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DRIVING ASSIST SYSTEM 

BACKGROUND OF THE INVENTION 

1. Field of the Invention 

s The present invention relates to driving assist systems and 

methods for assisting efforts by operators to operate vehicles in 
traveling. 

2. Description of the Background Art 

There is a need for a driving assist system, which can 

xo reliably assist effort by an operator to operate a vehicle, such as, 
an automobile, in traveling. Currently available automobiles are 
equipped with a data acquisition system and a controller 
including a processor. Such data acquisition system can 
continuously acquire data including information on vehicle state, 

15 such as current vehicle velocity, and information on 
environment in a field around the vehicle. The information on 
environment includes the presence of obstacles, including a 
leading vehicle ahead. 

Ideally, a driving assist system should avoid objection by 

20 the operator of the automobile in assisting the operator manual 
effort to reduce a risk of collision with an obstacle. However, if 
the vehicle should encounter risky environment involving the 
presence of an obstacle, the driving assist system should avoid 
collision with the obstacle. Thus, there is a need for a system, 

25 which performs assist so that the operator can continue driving 
comfortably without any objectionable feel due to such assist 

JP-A 10-211886, published on August 11, discloses a 
steering control system, which modulates power-assist for 
avoidance of collision upon recognition of a risk within a field 

30 around the vehicle. This steering control system computes the 
magnitude of risk with respect to each of obstacles within the 
field around the vehicle.. The magnitude of risk reflects the 
magnitude of impact upon collision with the obstacle. The 
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magnitude of Impact depends mainly upon the relative velocity 
between the vehicle and the obstacle. Accounting for this fact, 
the known system computes an individual risk R(i) from an 
angular location a(i) expressed in terms of an azimuth angle of 
5 an obstacle i, a distance or separation d(i) between the subject 
vehicle and the obstacle i, and an odd function of the relative 
velocity Vref(i) between the subject vehicle and the obstacle i. 
This odd function involves a term ofVref(i) n (n = a real number 
grater than or equal to 3). The individual risk R(i) is expressed 
10 as, 

R(l)m m^miL W (a(i)) Eq.l 
where, 

f(Vref(i)) = Odd function that involves a term of Vref(i) n (n * 3); 
d(i) = Distance (or separation) between the subject vehicle and 
the obstacle I; 

k = Constant to adjust the distance d(i); and 
W(a(i)) = Weighting component applied to the obstacle i 
appearing In the angular location a. 

The individual risks are summed vertorially to produce a 
resultant risk R, which is expressed as, 

# 

25 The known steering control system is capable of avoiding 
collision with the obstacle by assisting the operator to operate 
the vehicle laterally to clear the obstacle. However, this system 
appears to be insufficient to avoid objectionable feel,, which the 
operator might have. The operator may have such 

30 objectionable feel upon occurrence of a power assist, which has 
magnitude determined by the risk R, because a discrepancy 
exits between the risk R and a risk that is perceived by the 
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operator. When the vehicle encounters an obstacle, the 
magnitude of risk R is proportional to the cube of the relative 
velocity between the vehicle and the obstacle, whereas the 
magnitude of risk perceived by the operator is proportional to 
s the square of the relative velocity. 

It is therefore an object of the present invention to provide 
a driving assist system for assisting effort by an operator to 
operate a vehicle in traveling, which can assist the operator 
without causing the operator to have any objectionable feeL 
10 It is another object of the present invention to provide a 

simple and economical driving assist system for assisting effort 
by an operator to operate a vehicle in traveling. 

SUMMARY OF THE INVENTION 
The present invention provides, in one aspect thereof, a 
15 driving assist system for assisting effort by an operator to 
operate a vehicle in traveling, the driving assist system 
comprising: 

a data acquisition system acquiring data including 
information on vehicle state and information on environment in 
20 a field around the vehicle; 

a controller, mounted to the vehicle, for receiving the 
acquired data, for determining future environment in the field 
using the acquired data, for making an operator response plan in 
response to the determined future environment, which plan 
25 prompts the operator to operating the vehicle in a desired 
manner for the determined future environment, to determine 
command, and for generating the command; and 

at least one actuator, mounted to the vehicle, for 
prompting the operator in response to the command to 
30 operating the vehicle in the desired manner 

The present invention provides, in second aspect thereof, 
a driving assist system for assisting effort by an operator to 
operate a vehicle in traveling, the driving assist system 
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comprising: 

a data acquisition system acquiring data in volving 
information on the presence of a leading vehicle in a field around 
the vehicle; and 

s a controller, mounted to the vehicle, for receiving the 

acquired data, for determining a first extent to which the 
vehicles has approached to each other and a second extent to 
which the first extent might be influenced if a change in 
environment should occur, and for determining future 
io environment In the field based on the first and second extents. 

The present invention provides, in third aspect thereof, a 
driving assist system for assisting effort by an operator to 
operate a vehicle in traveling, the driving assist system 
comprising: 

is a data acquisition system acquiring data involving 

information on the presence of an obstacle in a field around the 
vehicle; 

a controller, mounted to the vehicle, for receiving the 
acquired data, for determining a risk which the obstacle would 
20 cause the operator to perceive, for allocating commands for 
prompting the operator to operating the vehicle longitudinally 
and laterally, and for generating the commands; and 

a plurality of actuators, mounted to the vehicle, to prompt 
the operator in response to the commands to operating the 
25 vehicle longitudinally and laterally. 

BRIEF DESCRIPTION OF THE DRAWINGS 
The present invention will be apparent from reading of the 
following description in conjunction with the accompanying 
drawings. 

30 Figure 1 is a control diagram showing control of a driving 

assist system. 

Figure 2 illustrate parameters relevant to a leading and 
following traffic scene. 
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Figure 3 is a block diagram showing hardware of a first 
exemplary implementation of the present invention. 

Figure 4 is a view illustrating how the hardware of the first 
implementation is arranged on an automobile. 

Figure 5 is a block diagram showing hardware of a second 
exemplary implementation of the present invention. 

Figure 6 is a view illustrating how the hardware of the 
second implementation is arranged on an automobile. 

Figure 7 is a flow diagram illustrating the operation of the 
first implementation. 

Figure 8 is a schematic view illustrating one example of 
current environment in a field around the subject vehicle, which 
is determined by the first implementation. 

Figure 9 is a schematic view illustrating a future 
environment in the field around the subject vehicle, which is 
predicted by the first implementation, and the optimum path. 

Figure 10 is a timing diagram illustrating an example of a 
variation curve of the optimum steering angle for the vehicle to 
track the optimum path. 

Figure 11 is a graphical representation of an example of 
steering reaction characteristic, which is provided at a moment 
of the curve of Figure 10, to prompt the vehicle operator to 
operating the steering wheel to the optimum steering angle. 

Figure 12 is a flow diagram illustrating the operation of a 
second exemplary implementation of the present invention. 

Figure 13 is a schematic view illustrating one example of 
current environment in a field around the subject vehicle, which 
is determined by the second implementation. 

Figure 14 is a schematic view illustrating a future 
environment in the field around the subject vehicle, which is 
predicted by the second implementation. 

Figure 15 is a timing diagram illustrating an example of a 
variation curve of the optimum vehicle velocity. 
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Figures 16A and 16B are timing diagrams illustrating the 
optimum coordinated variations of accelerator and brake pedal 
angular positions. 

Figure 17 is a graphical representation of one example of 
s accelerator pedal reaction characteristic, which is provided at a 
moment of the curve of Figure 16A, to prompt the vehicle 
operator to operating the accelerator pedal to the optimum 
pedal position. 

Figure 18 is a graphical representation of one example of 
10 brake pedal reaction characteristic, which is provided at a 
moment of the curve of Figure 16B, to prompt the vehicle 
operator to operating the brake pedal to the optimum pedal 
position. 

Figure 19 is a schematic view of a servo motor of an 
is accelerator reaction characteristic modulation actuator in 
operative relationship with an accelerator pedal. 

Figure 20 is a block diagram showing hardware of a third 
exemplary Implementation of the present invention. 

Figure 21 is a view illustrating how the hardware of the 
20 third implementation is arranged on an automobile. 

Figure 22 is a view illustrating an accelerator pedal 
reaction modulation mechanism. 

Figure 23 is a graph illustrating the relationship between 
accelerator pedal reaction force and stroke. 
25 Figure 24 is a flow diagram illustrating the operation of the 

third implementation. 

Figure 25 is a timing diagram illustrating how, in a traffic 
scene in Figure 2, a vehicle operator predicts a change in 
velocity of a leading vehicle. 
30 Figure 26 is a RP map. 

Figure 27 is another RP map. 

Figure 28 is a block diagram showing hardware of a fourth 
exemplary implementation of the present invention. 
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Figure 29 a Wew illustrating how the hardware of the third 
implementation is arranged on an automobile. 

Figure 30 is a flow diagram illustrating the operation of the 
fourth implementation. 
s Figure 31 is a graphical representation of a function 

determining a period of time that is used for prediction of a 
future environment in a field around the vehicle. 

Figure 32 is a flow diagram illustrating the operation of a 
first embodiment of a fifth exemplary implementation of the 
10 present invention. 

Figure 33 is a flow diagram illustrating the operation of a 
second embodiment of the fifth exemplary implementation of 
the present invention. 

Figure 34 is a graphical representation illustrating the 
is varying of risk perceived RP calculated as compared to risk 
actually perceived by operator during the first half and the 
second half when the vehicle is approaching to a leading vehicle. 

Figure 35 is a graphical representation illustrating risks 
RP 0 and RP 2 . 

20 Figure 36 is a map for parameters a, b. 

Figure 37 is a graphical representation illustrating risks 
RPo, RPi, and RP 2 . 

Figure 38 is a block diagram showing hardware of an 
eighth exemplary implementation of the present invention. 
25 Figure 39 is a view how the hardware of the eighth 

implementation is arranged on an automobile. 

Figure 40 is a flow diagram illustrating the operation of the 
eighth implementation. 

Figure 41 is a graphical representation of the varying of 
30 variance of separation for laser radar as compared to that for a 
CCD camera. 

Figure 42 is a graphical representation of the varying of 
variance of relative velocity for the laser radar as compared to 
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that for the CCD camera. 

Figure 43 is a graphical representation of the varying of 
variance of separation for the CCD camera with different kinds of 
obstacles. 

s Figure 44 is a graphical representation of the varying of 

variance of relative velocity for the CCD camera with different 
kinds of obstacles. 

Figure 45 is an accelerator pedal reaction command (Fa) 

map. 

10 Figure 46 is a brake pedal reaction command (Fb) map. 

Figure 47 is a steering reaction command (F s ) map. 
Figure 48 is a flow diagram illustrating the operation of a 
ninth exemplary implementation of the present invention. 

Figure 49 is a map illustrating the relationship between 
is RPk and operator inputs. 

Figure 50 is a map illustrating the relationship between 
ongitudinal and an operator input for opening the vehicle 
longitudinally. 

Figure 51 is a map illustrating the relationship between 
20 RPLaterai and an operator input for operating the vehicle laterally. 

Figure 52 is an accelerator pedal reaction command (Fa) 

map. 

Figure 53 is a brake pedal reaction command (F B ) map. 

Figure 54 is a steering reaction command (Fs) map. 
25 Figure 55 illustrates an accelerator pedal reaction 

characteristic according to the ninth implementation. 

Figure 56 illustrates a brake pedal reaction characteristic 
according to the ninth implementation. 

Figure 57 illustrates a steering reaction characteristic 
30 according to the ninth implementation. 

Figure 58 is a flow diagram illustrating the operation of a 
tenth exemplary implementation of the present invention. 

Figure 59 is a gain map according to the tenth 
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Implementation. 

DETAILED DESCRIPTION OF THE INVENTION 

Referring to Figure 1, a box 10 represents a vehicle 
operator, and a box 12 represents input devices. Input devices 
5 12 represent a steering wheel, an accelerator pedal and a brake 
pedal, which a motor vehicle 14 is usually equipped with. 

Vehicle operator 10 perceives the vehicle state through 
normal senses, represented by feedback line 16, and 
environment or situation in a field around the vehicle through 

io normal senses, represented by line 18. The operator applies 
manual effort, represented by line 20, to the accelerator pedal, 
represented by input device 12 sending a power request 
command to an engine controller, not shown, of vehicle 14 
through line 22. The operator applies manual effort, 

is represented by line 24, to the brake pedal, represented by input 
device 12, sending a brake request command to a braking 
system, not shown, of vehicle 14 through line 26. The operator 
applies manual steering effort, represented by 28, to the 
steering wheel, represented by input device 12 sending a 

20 steering request command to a steering system, not shown, of 
vehicle 14 through line 30. Input device 12 provides reaction 
force to the manual effort by the operator. 

The reference numeral 40 generally indicates a driving 
assist system for assisting effort by operator 10 to operate 

25 vehicle 14 in travelling. A data acquisition system (DAS) 42 
arranged on or off the vehicle, continuously acquires data 
including information on the state of vehicle 14 and information 
on environment in a field around the vehicle. The information on 
the vehicle state Includes velocity V f of vehicle 14. The 

30 information on environment includes the presence of obstacles, 
such as, a leading vehicle in the same lane and other adjacent 
vehicles in the same or other lanes, in the field around vehicle 14. 
DAS 42 provides the presence of each obstacle in terms of 
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angular location of the obstacle with respect to the subject 
vehicle 14 and relative velocity V r between the obstacle and the 
subject vehicle 14. DAS 42 may provide velocity of each of the 
obstacles. The information on environment also includes the 
5 presence of lane markings in the field around vehicle 14. DAS 
42 may provide the presence of lane markings in terms of lateral 
displacement of the subject vehicle 14. DAS 42 may provide 
weather condition in the way of the subject vehicle 14. 

Driving assist system 40 receives and uses the acquired 

io data, represented by line 44. A predicting module 46 uses the 
acquired data 44 to determine or predict future environment in 
the field around vehicle 14. Using the determined future 
environment as input 48, a plan making or building module 50 
makes or builds an operator-response plan, which prompts 

15 vehicle operator 10 to operating vehicle 14 in a desired manner 
for the predicted future environment To implement the 
operator-response plan, command or commands are 
determined. The determined command or commands are 
generated and provided to actuator(s) 52 through line 54. In 

20 response to the command(s), actuator(s) 52 prompts the 
vehicle operator 10 to operating the vehicle 14 in the desired 
manner. 

In embodiments, the actuator(s) 52 acts on input device 
12 through line 56 to modulate reaction characteristic to manual 

25 effort applied to the input device by the vehicle operator 10. In 
one embodiment, the future environment determined by 
predicting module 46 involves a risk which each of the 
determined obstacles would cause the vehicle operator 10 to 
perceive. Such risks are summed vertorially to produce a 

30 resultant risk. In this case, the planning module SO makes an 
operator-response plan that prompts operator 10 to operating 
vehicle in a desired manner to reduce the resultant risk. In 
another embodiment, the future environment further involves a 
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risk derived from the road condition. Such risk is determined by 
a lateral deviation of vehicle 14 from a lane and by curvature of 
the lane. In this embodiment, the planning module 50 makes an 
operator response plan that prompts operator 10 to operating 
5 vehicle in a desired manner to reduce the resultant risk 
superimposed by the risk derived from the road condition. In 
the embodiments, the operator-response plan made by planning 
module 50 includes the amount input to the actuator(s) 52. 

Building of the operator response plan must account for 

10 type of actuator(s) 52 employed. In one embodiment where the 
actuator(s) 52 is capable of modulating reaction characteristic to 
manual steering effort by operator 10, plan making module 50 
can make an operator-response plan, which prompts vehicle 
operator 10 to operating vehicle 14 laterally in such a manner as 

is to reduce the resultant risk. In another embodiment where the 
actuator(s) 52 is capable of modulating reaction characteristic to 
manual effort applied to the accelerator pedal along with 
modulating reaction characteristic to manual steering effort, 
plan making module 50 can make an operator-response plan, 

20 which prompts vehicle operator 10 to operating vehicle 14 
longitudinally and laterally in such a manner as to reduce the 
resultant risk. 

While the plan-making module 50 and predicting module 
4646 can be implemented in hardware or software, it is most 

25 likely implemented in software. In a software implementation, 
the modules 46 and 50 represent software instructions stored in 
a memory of a computer, and executed in the processor, and the 
acquired data stored in memory. In hardware implementations, 
the modules 46 and 50 are implemented in digital logic, 

30 preferably in an integrated circuit 

Before describing implementations in detail, it is helpful to 
begin with consideration of parameters that define relationship 
between the two adjacent vehicles on the same lane. Figure 2 
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illustrates the parameters relevant to two adjacent vehicles in 
travel on the same lane at different velocities. In the 
parameters, time to contact TTC and time headway THW are 
described later 

s The following description provides various 

implementations of the present invention. 

First Implementation of the Invention 
Figure 3 provides hardware, and Figure 4 illustrates how 
the hardware is arranged on an automobile 70. 
10 In Figure 3, the reference numeral 40A generally indicates 

a driving assist system. Driving assist system 40A includes an 
on board data acquisition system (DAS) 72. DAS 72, mounted 
to the vehicle 70, includes laser radar 74, a front camera 76, a 
rear right side (RRS) camera 78R, a rear left side (RLS) camera 
is 78L, and a source of vehicle speed 80. 

Referring to Figure 4, vehicle 70 has laser radar 74 
mounted to a front bumper thereof. Laser radar 74 may be 
mounted to a front grille of the vehicle. Laser radar 74 is of a 
type which can scan horizontally and laterally about 6 degrees to 
20 each side of an axis parallel to the vehicle longitudinal center! in e, 
propagates infrared pulses forward/y and receives the reflected 
radiation by an obstacle, such as, a rear bumper of a leading 
vehicle. Laser radar 74 can provide an angular location of the 
leading vehicle and a vehicle separation or a distance between 
25 the subject vehicle 70 and the leading vehicle. 

Front camera 76 is of a CCD type or a CMOS type and 
mounted to vehicle 70 in the vicinity of the internal rear view 
mirror to acquire image data of a region in front of the vehicle. 
The region covered by front camera 76 extends from a camera 
30 axis laterally to each side by about 30 degrees. 

RRS and RLS cameras 78R and 78L are, each, of a CCD 
type or a CMOS type. RRS camera 78R is mounted to vehicle 70 
in the vicinity of the internal rear right corner to acquire image 
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tfata of rear scenery including the adjacent lane on the 
right-hand side. RLS camera 78R is mounted to vehicle 70 in the 
vicinity of the interna/ rear left corner to acquire image data of 
rear scenery including the adjacent lane on the left-hand side. 

Source of vehicle speed 80 may obtain a measure of 
vehicle velocity by processing outputs from wheel speed sensors. 
The source of vehicle speed 80 may include an engine controller 
or a transmission controller, which can provide a signal 
indicative of the vehicle velocity. 

Driving assist system 40A includes a 
microprocessor-based controller 82. Controller 82 receives the 
acquired data from DAS 72. From source of vehicle speed 80, 
controller 82 receives information on the vehicle velocity V f . 
From laser radar 74, controller 82 receives information on 
vehicle separation D between the subject vehicle 70 and the 
adjacent leading vehicle. From the image data provided by front 
camera 76, controller 82 receives information on vehicle 
separation D between the subject vehicle 70 and each of 
vehicles in front From the image data provided by RRS and RLS 
cameras 78R and 78L, controller 82 receives information on the 
presence of vehicles, in the adjacent lanes, approaching from 
the rear, and it also receives information on extent to which each 
vehicle is approaching. From the image data provided by front 
camera 76, controller 82 receives information on the presence of 
lane markings on a road, information on the lateral position of 
the vehicle relative to the lane markings, and information on the 
angular position of the vehicle relative to the lane markings. 
Using such information, controller 82 determines current 
environment or situation in a field around vehicle 70. Using the 
current and past values of environment, controller 82 predicts or 
determines future environment in the field around vehicle 70 for 
building of an operator response plan. 

According to this implementation, the operator response 



• 14- P02NM-045US 0 1 -00643 

plan includes using the current and future environments to 
determine the optimum path for vehicle 70 to take, establishing 
the optimum steering angle for vehicle 70 to track the 
determined optimum path, and determining the optimum 
5 steering reaction characteristic for prompting the vehicle 
operator to operating the steering wheel to the optimum 
steering angle. Controller 82 determines the amount of 
command in response to the determined optimum steering 
reaction characteristics Controller 82 generates the command 

10 and applied it to a steering reaction modulation actuator 84. 

Steering reaction modulation actuator 84 includes a 
controller called a steering reaction characteristic modulator 86 
and a servo motor 88. Servo motor 88 is incorporated in the 
steering mechanism to modulate steering reaction characteristic 

is to operator steering effort. Modulator 86 controls output torque 
of servo motor 88 in response to the command from controller 
82 such that the steering reaction characteristic is adjusted to 
the reaction characteristic determined at controller 82. 

The flow diagram in Figure 7 illustrates a control routine of 

20 the first implementation of the present invention. The control 
routine, generally indicated at 100, is executed at regular time 
intervals of 50 milliseconds. 

At input box 102, the processor of controller 82 inputs 
acquired data by DAS 72. Specifically, the processor inputs 

25 vehicle velocity V f , an angular location of a leading vehicle and a 
vehicle separation D between the subject vehicle 70 and the 
leading vehicle, and relative relation between the subject vehicle 
70 and each of vehicles that are determined by RRS and RLS 
cameras 78R and 78L. 

30 At box 1 04, using the input data at box 1 02, the processor 

determines current environment in a field around vehicle 70. 
Particularly, the processor builds a current hazard map 
visualizing the possibility of collision of the subject vehicle 70 
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with each of other vehicles from current and past values of the 
acquired data at box 102. 

The possibility of collision with another vehicle lowers as 
the vehicle 70 separates from it. This possibility may be 
s expressed as a function of the reciprocal of vehicle separation or 
the reciprocal of the square of vehicle separation. If the relative 
velocity between the two vehicles can be neglected, isograms in 
the hazard map are concentric circles with the location of the 
vehicle as the center However, if the relative velocity between 

10 the two vehicles is not negligible, the isograms protrude in the 
direction of a vector of the relative velocity. 

Accounting for the vehicle lateral position from the lane 
marking and the road curvature has proven to be effective for 
enhanced building of a hazard map. The possibility of collision 

is grows big as the vehicle gets close to the lane marking 
separating the adjacent lanes or to the edge of a road. 
Weighting component is the least when the vehicle follows the 
centerline of a lane, and It gets great as the vehicle is close the 
lane marking or to the edge of road. With different weighting 

20 components, this collision possibility due to the lane marking 
and the before-mentioned collision possibility due to other 
vehicles are summed to provide enhanced hazard map. The 
weighting component applied to the collision possibility due to 
other vehicles is less than the weighting component applied to 

25 the collision possibility due to the lane markings. 

Figure 8 illustrates an example of a current environment 
expressed in terms of a hazard map at the moment (t = 0). The 
environment involves a vehicle traveling scenario on a road with 
three lanes. In the middle lane, the subject vehicle 70 is 

30 following a leading vehicle 120. Vehicle 70 is traveling at a 
vehicle velocity V f , while vehicle 120 is traveling at a vehicle 
velocity V a . As V f > V B , a vector originating at vehicle 120 
indicates the relative velocity between the two vehicles 70 and 
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120. In the left adjacent lane, a vehicle 122 is traveling at a 
vehicle velocity lower than V f . A vector originating at vehicle 122 
indicates the relative velocity between the two vehicles 70 and 
122. In the right adjacent lane, a vehicle 124 is traveling at a 
5 vehicle velocity higher than Vf. 

At the moment (t = 0), vehicle 122 closes a path 126 for 
vehicle 70 to take to the left adjacent lane. Thus, path 126 is not 
justified. Apparently, vehicle 124 is about to close space in the 
right adjacent lane, closing any path for vehicle 70 to take to the 

10 right adjacent lane. The vehicle separation between the two 
vehicles 70 and 120 allows vehicle 70 to stay in the middle lane. 
In this environment, what the vehicle operator has to do is to 
continue driving vehicle 70 in the middle lane. 

At box 106, after it has determined the current 

is environment, the processor predicts or determines future 
environments related to different future moments. The different 
future moments have different time values, respectively. The 
time values are arranged regularly with the largest time value 
being a predetermined time value of 5 seconds in this 

20 implementation. 

Figure 9 illustrates an example of one future environment 
expressed in terms of a hazard map at a future moment (t ~T). 

At this future moment (t = T), vehicle 122 opens a path 
126 for vehicle 70 to take to the left adjacent although the 

25 vehicle separation between vehicles 70 and 120 has reduced 
and vehicle 124 has closed space in the right adjacent lane. In 
this future environment, the path 126 is justified and may be set 
as the optimum path at the future moment (t = T). 

At box 108, the processor uses the predicted future 

30 environments to determine the optimum path at each of the 
moments. The following provides a description on what has 
been taken into account, in determining the optimum path at 
each moment The optimum path has been determined by 
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finding, in a hazard map for each moment, a provisional location, 
within an area around the location of vehicle 70, where the 
collision possibility is the local minimum. The processor checks 
on one after another of the provisional location and the adjacent 
s other locations whether or not lateral acceleration and yaw 
angular rate due to a change from the current location to the one 
of the locations falls within an acceptable range. If the 
provisional location has past the test, it is used as the optimum 
location. If not, any one of the adjacent other locations that has 

10 past the text is used as the optimum location. 

At box 110, the processor determines the optimum 
steering angle £ m to accomplish the optimum path, and the 
optimum steering reaction characteristic to prompt the vehicle 
operator to operating the steering wheel to the optimum 

is steering angled*. The optimum steering angled* may be 
determined by the steering characteristic and the optimum path. 
Figure 10 provides an example of a variation curve of the 
optimum steering angle £*. Figure 11 provides one example of 
steering reaction characteristic, which is provided at a moment 

20 of the variation curve of Figure 10, to prompt the vehicle 
operator to operating the steering wheel at the optimum 
steering angle 6 * at the moment The steering reaction 
characteristic of Figure 11 provides almost zero reaction force 
(F s = 0) at the optimum steering angle 6*. As the steering angle 

25 $ moves to the right out of this optimum steering angle 6*, the 
steering reaction force Fs increases at slope angle ofK S R> As the 
steering angle 6 moves to the left out of this optimum steering 
angle the steering reaction force F s increases at slope angle 

ofKsu 

30 The slope angles K S r and K SL are not fixed and determined 

by gradient of the collision possibility on one side in the vicinity 
of the optimum path and gradient on the opposite side in the 
vicinity thereof Such gradients can be given from isograms on 
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both sides of the optimum path within the predicted hazard 
maps (see box 106 in Figure 7). Operator steering effort to 
increase deviation from the optimum path to left or right is 
opposed by a reaction force determined by slope angle K SL or K S r. 
5 If a rapid change in the collision possibility exits on the right side 
of the optimum path, the slope angle K S r increases to provide 
increased reaction force. If a rapid change in the collision 
possibility exits on the left side of the optimum path, the slope 
angle Ksl increases to provide increased reaction force. This 

io steering reaction characteristic prompts the vehicle operator to 
operating the steering wheel to track the optimum path. 

At output box 112, the processor outputs command 
indicative of the determined reaction characteristic to steering 
reaction modulation actuator 84 (see Figure 3). The command 

is is applied to the modulator 86 of the actuator 84. Under the 
control of the modulator 86, servo motor 88 operates to provide 
the determined steering reaction characteristic to the manual 
steering effort applied to the steering wheel by the vehicle 
operator 

20 Second Implementation of the Invention 

Figure 5 provides hardware, and Figure 6 illustrates how 
the hardware is arranged on an automobile 70 B. 

In Figure 5, the reference numeral 40A generally indicates 
a driving assist system. Driving assist system 40B includes an 

25 on board data acquisition system (DAS) 72B. DAS 72B, 
mounted to the vehicle 70B, includes laser radar 74, a front 
camera 76, and a source of vehicle speed 80. DAS 72B is 
substantially the same as DAS 72 except the elimination ofRRS 
and RLS cameras 78R and 78L 

30 Referring to Figure 6, vehicle 70S has laser radar 74 

mounted to a front bumper thereof Front camera 76 of a CCD 
type or a CMOS type is mounted to vehicle 70B in the vicinity of 
the internal rear view mirror to acquire image data of a region in 
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front of the vehicle. Source of vehicle speed 80 may obtain a 
measure of vehicle velocity by processing outputs from wheel 
speed sensors. The source of vehicle speed 80 may include an 
engine controller or a transmission controller, which can provide 
s a signal indicative of the vehicle velocity. 

In Figure 5, driving assist system 40B includes a 
microprocessor based controller 82B. Controller 82B receives 
the acquired data from DAS 72B. From source of vehicle speed 
80, controller 82B receives information on the vehicle velocity Vf. 

10 From laser radar 74, controller 82B receives information on 
vehicle separation D between the subject vehicle 70B and the 
adjacent leading vehicle. From the image data provided by front 
camera 76, controller 82B receives information on vehicle 
separation D between the subject vehicle 70 B and each of 

is vehicles in front. From the image data provided by front camera 
76, controller 82B receives information on the presence of lane 
markings on a road, information on the lateral position of the 
vehicle relative to the lane markings, and information on the 
angular position of the vehicle relative to the lane markings. 

20 Using such information, controller 82B determines current 
environment or situation in a field around vehicle 70B. Using the 
current and past values of environment, controller 82B predicts 
or determines future environments in the field around vehicle 
70B for building of an operator response plan. 

25 According to this implementation, the operator response 

plan includes using the current and future environments to 
determine the optimum vehicle velocity for vehicle 70B to take, 
establishing the optimum pedal positions of an accelerator pedal 
and a brake pedal for vehicle 70 B to travel at the determined 

30 optimum vehicle velocity, and determining the optimum 
accelerator and brake reaction characteristics for prompting the 
vehicle operator to operating the accelerator pedal and brake 
pedal to the optimum pedal positions. Controller 82B 



•20- P02NM-045US 01-00643 

determines the amount of accelerator pedal reaction command 
in response to the determined optimum accelerator reaction 
characteristic. Controller 823 also determines the amount of 
brake pedal reaction command in response to the determined 
5 optimum brake reaction characteristic. Controller 82B 
generates the accelerator reaction command and applies it to an 
accelerator reaction modulation actuator 140. Controller 82B 
generates the brake reaction command and applies it to a brake 
reaction modulation actuator 142. 

10 Accelerator reaction modulation actuator 140 includes a 

controller called an accelerator pedal reaction characteristic 
modulator 144 and a servo motor 146. Brake reaction 
modulation actuator 142 includes a controller called a brake 
pedal reaction characteristic modulator 148 and a brake booster 

15 ISO. 

Referring also to Figure 19, servo motor 146 is 
incorporated into a link mechanism of an accelerator pedal 152 
to modulate accelerator reaction characteristic to operator effort 
to depress accelerator pedal 152. Modulator 144 controls 

20 output torque of servo motor 146 in response to the command 
from controller 82B such that the accelerator reaction 
characteristic is adjusted to the reaction characteristic 
determined at controller 82B. 

Referring back to Figures 5 and 6, brake booster 150 can 

25 vary a brake assist to modulate brake reaction characteristic to 
operator effort to depress a brake pedal 154. Modulator 148 
controls brake booster 150 in response to the command from 
controller 82B such that the brake reaction characteristic is 
adjusted to the reaction characteristic determined at controller 

30 82B. 

The flow diagram in Figure 12 illustrates a control routine 
of the second implementation of the present invention. The 
control routine, generally indicated at 160 f is executed at 
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regular time intervals of 50 milliseconds. 

At input box 162, the processor of controller 82B inputs 
acquired data by DAS 72B. Using the image data from front 
camera 76, the processor determines the position of lane 
5 markings and the position of each of vehicles in front relative to 
the lane markings. Based on the relative position of each of the 
vehicles in front to the lane markings, the processor picks up a 
leading vehicle on the same lane as a target to be monitored by 
laser radar 74 for the subsequent control. The processor Inputs 

10 an angular location of the leading vehicle, and a vehicle 
separation D between the subject vehicle 70B and the leading 
vehicle from laser radar 74. The processor receives vehicle 
velocity Vf from source of vehicle speed 80. 

At box 164, using the acquired data received at box 162, 

is the processor determines current environment in a field around 
vehicle 70B. Particularly, the processor builds a current hazard 
map visualizing the possibility of collision of the subject vehicle 
70B with the following vehicle from current and past values of 
the acquired data. 

20 The possibility of collision with the leading vehicle lowers 

as the vehicle 70B separates from it This possibility may be 
expressed as a function of the reciprocal of vehicle separation or 
the reciprocal of the square of vehicle separation. If the relative 
velocity between the two vehicles can be neglected, isograms in 

25 the hazard map are concentric circles with the location of the 
vehicle as the center. However, if the relative velocity between 
the two vehicles is not negligible, the isograms protrude in the 
direction of a vector of the relative velocity. 

Figure 13 illustrates an example of a current environment 

30 expressed in terms of a hazard map at the moment (t - O). The 
environment involves a vehicle traveling scenario within a lane 
on road. A leading vehicle 120 exists in the same lane. Vehicle 
70B is traveling at a vehicle velocity Vt, while vehicle 120 is 
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traveling at a vehicle velocity V a . As V f > V d , a vector originating 
at vehicle 120 indicates the relative velocity between the two 
vehicles 70B and 120. At the moment (t - 0), there is a 
sufficient vehicle separation. 
s At box 166, after it has determined the current 

environment, the processor predicts or determines future 
environments related to different future moments. The different 
future moments have different time values, respectively. The 
time values are arranged regularly with the largest time value 

10 being a predetermined time value of 5 seconds in this 
implementation. 

Figure 14 illustrates an example of one future 
environment expressed in terms of a hazard map at a future 
moment (t =T). At this future moment (t = T), the vehicle 

is separation is not sufficient. 

At box 168, the processor uses the predicted future 
environments to determine the optimum path at each of the 
moments. The following provides a description on what has 
been taken into account in determining the optimum path at 

20 each moment The optimum path has been determined by 
finding, in a hazard map for each moment, a provisional location, 
within an area around the location of vehicle 70B, where the 
collision possibility is the local minimum. The processor checks 
on one after another of the provisional location and the adjacent 

25 other locations whether or not longitudinal acceleration due to a 
change from the current location to the one of the locations falls 
within an acceptable range. If the provisional location has past 
the test, it is used as the optimum location. If not, any one of 
the adjacent other locations that has past the text is used as the 

30 optimum location. In the second implementation, the lateral 
control is out of consideration. Hence, the optimum path is 
found in the same lane so that it may be taken by taken by 
vehicle 70B by varying the vehicle velocity. 
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/\t box 1 70, the processor determines the optimum vehicle 
velocity V to accomplish the optimum path, the optimum 
accelerator and brake pedal positions e A and e B *, and 
accelerator and brake pedal reaction characteristics. Figure 15 
5 provides an example of a variation curve of the optimum vehicle 
velocity V*. Figures 16A and 16B provide one example of a 
coordinated variations of optimum accelerator and brake pedal 
positions e A and de* to accomplish the optimum velocity 
variation shown in Figure 15. In Figures 16A and 163, during 

10 period of time Ti and T2, the optimum accelerator pedal position 
0/ slightly reduces to gradually slow vehicle 70B down. During 
the subsequent period of time T 2 and T 3 , the optimum 
accelerator pedal position e A * has reduced to zero and the 
optimum brake pedal position 8 B * increases temporally to 

15 effectively decelerate vehicle 70B, Finally, during period of time 
T 3 and T 4 , the optimum accelerator pedal position e A accelerate 
vehicle 70B until it can beautifully follow the leading vehicle. 

According to the second implementation, modulating 
accelerator and brake pedal reaction characteristic has 

20 accomplished the desired coordinated variations of the optimum 
accelerator and brake pedal positions e A * and e 0 *. 

Figure 17 provides one example of accelerator pedal 
reaction characteristic, which is provided at a moment of the 
variation curve of Figure ISA, to prompt the vehicle operator to 

25 operating the accelerator pedal at the optimum accelerator 
pedal position d* at the moment. The accelerator reaction 
characteristic of Figure 17 is recessed from the usual reaction 
characteristic, as indicated by slope angle K A0 , at the optimum 
accelerator pedal position e A *. As accelerator pedal position e A 

30 moves to the right out of this optimum accelerator pedal position 
e A m , the accelerator pedal reaction force F A increases at slope 
angle of K AF . As accelerator pedal position e A moves to the left 
out of this optimum accelerator pedal position e", the 
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accelerator pedal reaction force F A increases at slope angle of 
Kab. 

The slope angles K AF and K AB are not fixed and determined 
by gradient of the collision possibility if deviated longitudinally 
5 from the optimum position. Such gradients can be given from 
isograms around the optimum position within the predicted 
hazard maps (see box 166 in Figure 12). Operator manual effort 
to operate the accelerator pedal in a direction to increase 
longitudinal deviation from the optimum position is opposed by a 

10 reaction force determined by slope angle K A f or K AB . If a rapid 
change in the collision possibility exits on the leading side of the 
optimum point, the slope angle increases to provide 
increased reaction force. If a rapid change in the collision 
possibility exits on the trailing side of the optimum point, the 

is slope angle K AB increases to provide increased reaction force. 
This reaction characteristic prompts the vehicle operator to 
operating the accelerator pedal to track the optimum path. 

Figure 18 provides one example of brake pedal reaction 
characteristic, which is provided at a moment of the variation 

20 curve of Figure 16B, to prompt the vehicle operator to operating 
the brake pedal at the optimum brake pedal position e B * at the 
moment The brake pedal reaction characteristic of Figure 18 is 
recessed from the usual reaction characteristic, at the optimum 
brake pedal position e A . As brake pedal position e B moves to the 

25 right out of this optimum brake pedal position ef, the brake 
pedal reaction force F B increases at slope angle of K B p. As brake 
pedal position e B moves to the left out of this optimum brake 
pedal position g t m , the brake pedal reaction force F B increases at 
slope angle of K BB . 

30 The slope angles K BF and K BB are not fixed and determined 

by gradient of the collision possibility if deviated longitudinally 
from the optimum position. Such gradients can be given from 
isograms around the optimum position within the predicted 
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hazard maps (see box 166 in Figure 12). Operator manual effort 
to operate the brake pedal in a direction to increase longitudinal 
deviation from the optimum position is opposed by a reaction 
force determined by slope angle K BF or K BB . If a rapid change in 
the collision possibility exits on the leading side of the optimum 
point, the slope angle K BB increases to provide increased 
reaction force. If a rapid change in the collision possibility exits 
on the trailing side of the optimum point, the slope angle K B r 
increases to provide increased reaction force. This reaction 
characteristic prompts the vehicle operator to operating the 
brake pedal to track the optimum path. 

At output box 172, the processor outputs first command 
indicative of the determined accelerator pedal reaction 
characteristic and second command indicative of the determined 
brake pedal reaction characteristic. The first command is 
applied to the modulator 144 of the actuator 140. Under the 
control of the modulator 144, servo motor 146 operates to 
provide the determined accelerator pedal reaction characteristic 
to the manual effort applied to the accelerator pedal by the 
vehicle operator. The second command is applied to the 
modulator 148 of the actuator 142. Under the control of the 
modulator 148, brake booster 150 operates to provide the 
determined brake pedal reaction characteristic to the manual 
effort applied to the brake pedal by the vehicle operator. 

Third Implementation of the Invention 
[First Embodiment] 

Figure 20 provides hardware and Figure 21 illustrates how 
the hardware is arranged on an automobile 70C. 

In Figure 20, the reference numeral 40C generally 
indicates a driving assist system. Driving assist system 40C 
includes an on board data acquisition system (DAS) 72C. DAS 
72C, mounted to the vehicle 70C, includes laser radar 74 and a 
source of vehicle speed 80. The laser radar 74 and source of 
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vehicle speed 80 are mounted to vehicle 70C in the same 
manner as the second implementation (see Figures 5 and 6). 

In Figure 20, driving assist system 40C includes a 
microprocessor based controller 82C. Controller 82C receives 
s the acquired data from DAS 72C in the same manner as 
controller 82B receives the acquired data from DAS 72B. From 
source of vehicle speed 80, controller 82C receives information 
on the vehicle velocity V>- From laser radar 74, controller 82B 
receives information on vehicle separation D between the 

10 subject vehicle 70C and the adjacent leading vehicle and relative 
velocity V r between them. 

Controller 82C determines, by calculation for example, a 
first extent to which vehicle 70C has approached to a leading 
vehicle from vehicle separation D between the two vehicles and 

is relative velocity V r between them. Controller 82C determines 
current environment in a field around vehicle 70C from the first 
extent Further, controller 82c predicts how the current 
environment will progress in future and uses the result of such 
prediction to output command to a controller called an 

20 accelerator pedal reaction characteristic modulator 144 of an 
accelerator reaction modulation actuator 140. Modulation 
actuator 140 includes a servo motor 146 and a stroke sensor 
178. 

Referring also to Figure 22, servo motor 146 and stroke 
25 sensor 178 are incorporated in a link mechanism of an 
accelerator pedal 1 52. Stroke sensor 1 78 senses the magnitude 
of operator manual effort applied to accelerator pedal 1 52. The 
output of stroke sensor 178 is fed to modulator 144. In 
response to command from modulator 144, servo motor 146 
30 produces torque, modulating reaction force to the manual effort 
applied by vehicle operator to accelerator pedal 152. 

Controller 82C receives information on vehicle separation 
D between vehicle 70C and a leading vehicle, information on 
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relative velocity V r between the vehicles, and information on 
vehicle velocity V f of vehicle 70C from the acquired data by DAS 
72C. Using the received information, controller 82C determines, 
by calculation for example, the first extent (-a first risk 
5 category) to which vehicle 70C has approached the leading 
vehicle, and a second extent (= a second risk category) to which 
vehicle 70C might be influenced due to a predictable change in 
movement of the leading vehicle. Controller 82C uses the first 
and second extents to predict future environment expressed in 

10 terms of risk perceived (RP), determines an accelerator pedal 
reaction command A F based on the RP. Command A F is applied 
to modulator 144. Modulator 144 controls servo motor 146 in 
response to command A F, thus modulating reaction force versus 
stroke characteristic of accelerator pedal 152. 

is In Figure 23, a set of two parallel line segments 

interconnected by another set of two parallel line segments to 
enclose an area dotted illustrate usual reaction force 
characteristic. Under this condition, no additional force is 
applied to modulate the reaction force characteristic. With the 

20 accelerator pedal reaction modulation, this usual reaction force 
characteristic is elevated by A F over the whole range of stroke. 
AsAF is proportional to RP, the vehicle operator can perceive 
through accelerator pedal 162 the current environment and 
predicted future environment in the field around vehicle 70C. 

25 The flow diagram in Figure 24 illustrates a control routine 

of the third implementation. The control routine, generally 
indicated at 180, is executed at regular time intervals of SO 
milliseconds. 

At input box 182, the processor of controller 82C Inputs 
30 acquired data by DAS 72C. What are read at box 182 include 
velocity V f of vehicle 70C, vehicle separation D between vehicle 
70C and a leading vehicle, relative velocity V r between the 
vehicles, and velocity V & of the leading vehicle. 



1 



_ J [ 



-28- P02NM-045US 01-00643 

At box 184, the processor calculates time to contact (TTC) 
and time headway (THW). 

We introduced the notion of TTC to quantify a first extent 
to which the subject vehicle 70C has approached to a leading 
5 vehicle in the traffic scene illustrated in Figure 2. This first 
extent is introduced to represent how much the vehicle 70C has 
approached to the leading vehicle. 

TTC is expressed as, 

10 TTC « D/V r Eq. 3 

where, 

D = Vehicle separation between the vehicles in the traffic scene 

illustrated in Figure 2 

V r - Relative vehicle velocity V f - V a 

15 

In the traffic scene of Figure 2, most operators perceived risk 
and operated the following vehicles for deceleration well before 
TTC becoming less than 4 seconds. We considered that, risk, 
which each vehicle operator perceived, prompted the operator 

20 to operate the vehicle longitudinally to resume sufficient vehicle 
separation. The shorter TTC, the greater the magnitude of risk, 
which the leading vehicle causes the vehicle operator of the 
following vehicle to perceive, is. However, there is some 
discrepancy between TTC and risk perceived by the vehicle 

25 operator. Thus, TTC only is insufficient to quantify or express 
the risk perceived by operator. 

The presence of such discrepancy may be confirmed by 
considering a traffic scene where the relative velocity V r between 
the leading and following vehicles is zero. In this case, TTC is 

30 infinite irrespective of how short the vehicle separation D is. 
Apparently, the risk perceived by the operator of the following 
vehicle varies with different distances of vehicle separation D. 
The shorter the vehicle separation D, the bigger the risk 
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perceived by the operator is. This is because the vehicle 
operator accounts for the magnitude of influence on TTC due to 
unpredictable drop in velocity V a of the leading vehicle. 

With reference to Figure 25, this section provides a further 
5 description on the above issue. Immediately before the moment 
t = 0, the relative velocity V r between the leading and following 
vehicles is positive and fixed, and TTC has been updated by 
calculating the equation 3. In operating the following vehicle, 
the operator keeps on predicting the magnitude of an 

10 unpredictable change, particularly, a drop, in velocity V a of the 
leading vehicle, which might occur at the moment At after. 
Accurate prediction of V a at the moment At after is impossible, 
however. Thus, the operator tries to predict the velocity V a with 
considerable dispersion as illustrated in Figure 25. In Figure 25, 

is if, at the moment At, the velocity V a drops to V 2 from V Jf TTC 
becomes shorter than expected At before, resulting in an 
increase in risk than expected. Predication of such increase in 
TTC from the current and past values of TTC is difficult. 

To overcome this difficulty, according to the third 

20 implementation, we introduced the notion of time headway 
(THW). THW is a physical quantity quantifying a second extent 
to which TTC might be influenced if a change in velocity V a of a 
leading vehicle should occur. The second extent is introduced to 
represent how much a change in relative velocity v r , if any in 

25 immediate future, might influence TTC. THW is expressed as, 

THW = D/V a Eq. 4 

or 

THW = D/Vf Eq. 5 

30 

THW is a measure of a timer that is set to count when the 
leading vehicle reaches .a point on a road and will be reset 
subsequently when the following vehicle will reach the same 
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point The longer THW, the smaller the second extent is. That is, 
when THW is long, the first extent is not greatly influenced due 
to a change in V a , if any, of the leading vehicle, so that a change 
in TTC is sufficiently small. 
5 Using THW is to quantify the influence caused due to a 

future change in V a . In his respect, the use of THW determined 
by V 3 is recommendable rather than the other THW determined 
by V f .. Besides, the former represents the risk that is perceived 
by the operator with better accuracy than the latter does. 

10 However, if a value of V a that is given by a measure of V f and a 
measure of V r is less reliable than a value of V f , the use of THW 
determined by V r is recommendable. In the traffic scene where 
V a = Vf, the equations 4 and 5 are equally recommendable. 

With continuing reference to the flow diagram in Figure 24, 

is after calculating TTC and THW at box 184, the logic goes to box 
186. At box 186, the processor uses TTC and THW to create a 
physical quantity that is indicative of risk perceived (RP) by the 
vehicle operator. We introduced the notion of RP, which we call 
"Risk Perceived", to represent the magnitude of risk, which a 

20 vehicle operator would perceive upon predicting future 
environment in a field around the vehicle. In the traffic scene 
illustrated in Figure 2, the environment in the field around 
vehicle involves the presence of a leading vehicle. In creating 
the RP, we introduced the reciprocal of TTC (1 /TTC) as the first 

25 extent and the reciprocal of THW (1/THW) as the second extent 
According to the third implementation, they are summed to 
produce the RP, which is expressed as, 

RP = a(l/THW) + b(l/TTC) Eq. 6, 

30 

where b and a (b > a) are parameters weighting the first extent 
(1/TTC) and the second extent (1/THW), respectively, such that 
the second extent (1/THW) is less weighted than the first extent 
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(1/TTC) is. The values ofb and a are optimized after accounting 
for a statistics of values of THW and TTC collected in the traffic 
scene including leading and trailing vehicles. In this 
implementation, b = 8 and a = 1. 

5 

The equation 3 clearly reveals that time to contact TTC is a 
first risk category or component indicating how long it takes for 
a following vehicle to contact with a leading vehicle if it is 
assumed that the relative velocity V r between the vehicles is 

10 constant. The equations 4 and 5 reveal that time headway THW 
is a second risk category or component indicating how long it 
takes for the following vehicle to arrive at the point, which the 
leading vehicle has arrived at, from the instance of arrival of the 
leading vehicle if it is assumed that the relative velocity V r will be 

15 subject to a drop. Using the first and second risk categories TTC 
and THW, defined by current values of D, V f , V a and V r , in the 
equation 6 yields risk perceived RP, which represents a 
predicted future environment in the field around the vehicle. 
This operation is nothing but predicting a future environment in 

20 a field around the vehicle using current values ofD, V f , V a and V r . 

The RP can provide quantitative expression as to how 
much a following vehicle has approached to a leading vehicle 
continuously over a range from its separating from the leading 
vehicle to its approaching to same. The more it increases, the 

25 more strongly an operator perceives risk of excessively 
approaching to the leading vehicle in immediate future. 

The RP map in Figure 26 illustrates, within a THW- (1/TTC) 
plane, isograms of RP drawn based on the equation 6. The 
horizontal axis represents THW. The horizontal axis is a 

30 horizontal line originating at a value of 0.0 on the vertical axis 
and extending in the rightward direction. The vehicle is 
assumed to be following a leading vehicle with an unaltered 
relative velocity V r . In Figure 26, the vehicle is operating at 
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some operating point on the horizontal axis. The further the 
operating point on the horizontal axis is distant from the vertical 
axis in the rightward direction, the further the vehicle is 
separated from the leading vehicle. The vertical axis represents 
5 the reciprocal of TTC (1/TTC). Now, when the vehicle is 
assumed to start approaching to the leading vehicle due to a 
drop in relative velocity V r (= V f - V a ). In Figure 26, the 
operating point at which the vehicle is operating moves from the 
operating point on the horizontal axis as directed by an 

10 approaching mot/on vector. This motion vector is a resultant 
vector of a horizontal vector component oriented in the leftward 
direction and a vertical vector component in the upward 
direction. Next, when the vehicle is assumed to start separating 
from the leading vehicle due to an increase in relative velocity V r . 

is In Figure 26, the operating point at which the vehicle is 
operating moves from the operating point on the horizontal axis 
as directed by a separating motion vector. This motion vector is 
a resultant vector of a horizontal vector component oriented in 
the rightward direction and a vertical vector component in the 

20 downward direction. 

The operating point approaching to the leading vehicle 
moves through an upper section of the isograms of RP in a 
direction increasing RP, while the separating point separating 
the leading vehicle moves through a lower or the remaining 

25 section of the isograms RP in a direction reducing RP. The upper 
and lower sections are interconnected at the horizontal axis. In 
the upper section, lines of the isograms extend from vertically 
spaced generally equidistant start points on the right edge of the 
RP map to differently spaced intermediate points on the 

30 horizontal axis, respectively. The intermediate points are 
arranged such that a distant between the adjacent two points of 
them becomes narrower and narrower as the THW becomes 
shorter and shorter. The tines of the isograms curve 
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downwardly from the start points to the intermediate points to 
define a range of the highest or higher values of the RP, Such 
range is reached at a value of the reciprocal of TTC (1/TTC), 
which becomes smaller as the value of THW becomes smaller. 
5 Further, with the same value of THW, the higher the reciprocal of 
TTC (1/TTC), the higher the RP is. Besides, with the same value 
of the reciprocal of TTC (1/TTC), the shorter the THW, the higher 
the RP is. In the lower section, lines of the isograms extend from 
vertically spaced generally equidistant start points on a vertical 

io line, not illustrated, near the left edge of the RP map to the 
differently spaced intermediate points on the horizontal axis and 
a point on the right edge of the map, respectively. The lines of 
the isograms curve upwardly from the start points to the 
intermediate points to define a range of the lowest or zero value 

is oftheRP. 

With continuing reference to the flow diagram of Figure 24, 
after calculation of the RP at box 186, the logic goes to box 188. 

At box 188, the processor uses the RP to determine an 
accelerator pedal reaction command AF, which is expressed as, 

20 

AF = K • RP Eq. 7 

where, 

K = Constant determined for adjustment. 

25 As readily seen from the RP map in Figure 26, the RP takes 

various values continuously over different values of THW and 
1/TTC of environment in a space around the vehicle. The RP 
determines the command AF as expressed by the equation 7, 
which, in turn, determines accelerator pedal reaction force. This 

30 makes it possible for the operator to continuously know 
information how much the vehicle has approached to the leading 
vehicle. 

At the next box 190, the processor outputs the command 
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AF and applies it to the accelerator pedal reaction characteristic 
modulator 144 (see Figure 20). 

The preceding description on the equation 6 and the RP 
map in Figure 26 clearly reveals that the RP takes various values 
s continuously even if the first extent (1/TTC) and/or the second 
extent (1/THW) are subject to a change. Such smooth and 
continuous change in the RP makes it possible to vary 
accelerator pedal reaction force accordingly. Hence, the 
operator can recognize varying of environment through smooth 

10 and continuous change in accelerator pedal reaction force. 
[Second Embodiment] 

The second embodiment of the third implementation is 
substantially the same as the first embodiment thereof except 
the manner of producing the RP. In the first embodiment, the RP 

is was expressed as the equation 6. In this second embodiment, 
RP is expressed as, 



RP = max {a/THW, b/TTC} Eq. 8 

20 where, b and a (b > a) are parameters weighting the first extent 
(1/TTC) and the second extent (1/THW), respectively, such that 
the second extent (1/THW) is less weighted than the first extent 
(1/TTC) is. The values ofb and a are optimized after accounting 
for a statistics of values of THW and TTC collected in the traffic 

25 scene including leading and trailing vehicles. In this 
implementation, b = 8 and a = 1. 



According to the equation, the larger one of a/THW and b/TTC is 
selected and used as RP, 
30 The RP map in Figure 27 illustrates, within a THW- (1/TTC) 

plane, isograms of RP drawn based on the equation 8. The 
horizontal axis represents THW. The horizontal axis is a 
horizontal line originating at a value of 0.0 on the vertical axis 
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and extending in the rightward direction. The vertical axis 
represents the reciprocal of TTC (1/TTC). 

Prior to a further description on the RP map in Figure 27, 
this section provides a brief review of the equation 6 and its RP 
5 map in Figure 26. When the vehicle is assumed to be separating 
from the leading vehicle, the relative vehicle velocity V r is /ess 
than zero, and the term 1/TTC of the equation 6 is less than zero 
because TTC corresponds to D/V n According to the equation 6, 
with the same THW, the RP takes various small values for 

10 different minus values of 1/TTC. Because the RP takes such 
small values, the accelerator pedal reaction command A F takes 
small values, accordingly. 

According to the equation 8 and the RP map in Figure 27, 
the above-mentioned influence on the RP when the vehicle is 

is separating from the leading vehicle is eliminated. This is 
because the equation 8 selects a /THW as RP when b/TTC is less 
than 0. With the same THW, the RP does not drop below a value 
that is determined by a current value of THW over different 
minus values of the reciprocal of 1/TTC. This provides a gradual 

20 reduction in RP when the vehicle is separating from the leading 
vehicle. 

According to the third implementation, the RP is indicative 
of the magnitude of risk actually felt or perceived by an operator 
of a vehicle when, for example, involved in a traffic scene 

25 including a leading vehicle ahead. In the first embodiment, the 
RP is expressed by the equation 6 and illustrated in Figure 26. In 
the second embodiment, the RP is expressed by the equation 8 
and illustrated in Figure 27. The RP determines the accelerator 
pedal reaction command AF as expressed by the equation 7. The 

30 command AF determines the magnitude of accelerator pedal 
reaction force to the manual effort applied to the accelerator 
pedal 152 by the operator. The equation 7 clearly indicates that 
the accelerator pedal reaction force is proportional to the RP. 
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77? us, tfte accelerator pedal reaction force increases reflecting an 
increase in the risk actually perceived by the vehicle operator. 
The increase in accelerator pedal reaction force is transmitted to 
and felt by the operator through one's foot on the accelerator 
5 pedal 152, prompting the operator to allowing the accelerator 
pedal 152 to return to the released position. In other words, the 
operator is prompted to operating the vehicle longitudinally in a 
manner to reduce the risk, which the operator is caused to 
perceive by the leading vehicle. 

10 It will be appreciated, as an advantage, that the operator 

normally perceives the risk when the accelerator pedal reaction 
force is increased to prompt the operator to operating the 
vehicle in such a direction as to eliminate or at least reduce the 
risk. Accordingly, the operator is prompted to operating the 

is vehicle in the direction with little objection to such driving assist. 

Expressing in concrete terms, upon recognition of an 
increase in the RP by an increase in accelerator pedal reaction 
force, the operator is inspired to allow the accelerator pedal 152 
to move toward the released position. The increase in 

20 accelerator pedal reaction force causes the operator foot to 
move automatically to release the accelerator pedal 152, 
prompting the operator to operating the vehicle in the direction 
to eliminate or at least reduce the risk. Besides, the increase in 
accelerator pedal reaction force discourages the operator from 

25 further depressing the accelerator pedal 152 to reduce the 
vehicle separation. 

In the First embodiment of the third implementation, the 
equation 6 is used to produce RP. As is readily seem from the RP 
map in Figure 26, the RP varies continuously, so that the 

30 accelerator pedal reaction command AF and the accelerator 
pedal reaction force vary continuously, accordingly. Via such 
accelerator pedal reaction force, the operator is kept being 
informed of varying of environment in a field around the vehicle 
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with different values of 1/TTC and THW. 

In the second embodiment of the third implementation, 
the equation 8 is used to produce RP. As illustrated in Figure 27, 
the RP does not vary rapidly when the vehicle is separating from 
s the leading vehicle where the values of 1/TTC are less than zero 
and very small. Because the RP does not vary rapidly, it is 
possible to control accelerator pedal reaction force, without any 
sudden change. 

In the embodiments according to the third implementation, 

10 time to contact TTC and time headway THW are provided by 
calculation of simple equations involving easily measurable or 
obtainable physical quantities like velocity V f of trailing vehicle, 
velocity V a of leading vehicle, and vehicle separation D. This 
works in suppressing an increase, if any, in number of 

is component parts of a driving assist system. 

As has been described in connection with the equations 6 
and 8, the parameter b is greater than the parameter a (b > a) 
such that the second extent (l/THW) is less weighted than the 
first extent (1/TTC) is. In the embodiments of the third 

20 implementation, the first extent (i/TTC) is heavily weighted 
than the second extent (l/THW) in producing the RP. 

Fourth Implementation of the Invention 
Figure 28 provides hardware and Figure 29 illustrates how 
the hardware is arranged on an automobile 70D. 

25 In Figure 28, the reference numeral 40D generally 

indicates a driving assist system. Driving assist system 40D is 
substantially the same, in hardware, as driving assist system 
40C except the provision of a front camera 76 and a 
communication tool 208. Thus, the like reference numerals are 

30 used to indicate like parts or portions throughout Figures 20, 21, 
28 and 29. The front camera 76 and communication tool 208 
are parts of an on board data acquisition system (DAS) 72D 
mounted to the vehicle 70D. DAS 72D further includes laser 
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radar 74 and a source of vehicle speed 80. DAS 72D is 
substantially the same as DAS 72B shown in Figure 5 except the 
provision of communication tool 208 mounted to the vehicle 70D. 
Communication tool 208 includes an infrastructure receiver or a 
GPS receiver and receives information from an antenna 210 
attached to the vehicle 70D. In the same manner as the second 
implementation (see Figures 5 and 6), the laser radar 74, front 
camera 76, and source of vehicle speed 80 are mounted to the 
vehicle 70D. 

Driving assist system 40D includes a microprocessor 
based controller 82D. Controller 82D receives information on 
vehicle separation D between vehicle 70D and a leading vehicle, 
information on relative velocity V r between the vehicles, and 
information on velocity V f of vehicle 70D from the acquired data 
by laser radar 74 and source of vehicle speed 80 of DAS 72D. 
Controller 82D also receives information on the leading vehicle 
after processing the image data captured by front camera 70. 
The processing of the image data includes filtering and image 
pattern recognition. Such processing may be carried out within 
or outside of controller 82 D. Controller 82 D receives current 
and future information ahead of vehicle 70D from 
communication tool 208. Such information captured by 
communication tool 208 via antenna 210 will be later described. 

Using the received information, controller 82D determines, 
by calculation for example, the first extent 1/TTC to which 
vehicle 70D has approached the leading vehicle. Controller 82D 
determines a period of time v 0 in response to the current value 
of the first extent 1/TTC, and predicts a future value of the first 
extent the determined period of time after as a future 
environment in a field around vehicle 70 D. In this 
implementation, a risk perceived RP by operator expresses the 
future environment Based on the RP, controller 82D 
determines an accelerator pedal reaction command A F. 
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Command A F is applied to an accelerator pedal reaction 
characteristic modulator 144. Modulator 144 controls a servo 
motor 146 in response to command 4 F, thus modulating 
reaction force versus stroke characteristic of an accelerator 
s pedal 152. 

The flow diagram in Figure 30 illustrates a control routine 
of the fourth implementation. The control routine, generally 
indicated at 220, is executed at regular time intervals of 50 
milliseconds, 

10 At input box 222, the processor of controller 82D inputs 

acquired data by DAS 72D. What are read at box 222 include 
velocity V f of the vehicle 70D, vehicle separation D between 
vehicle 70D and a leading vehicle, relative vehicle velocity V r 
between the vehicles, and velocity V a of the leading vehicle. 

is At box 224, the processor calculates time to contact (TTC) 

that is expressed by the equation 3. As mentioned before in the 
third implementation, time to contact TTC is introduced to 
quantity a first extent to which the subject vehicle 70D has 
approached to a leading vehicle in the traffic scene illustrated in 

20 Figure 2. As is readily seen from the equation 3, time to contact 
TTC indicates how much it takes for the subject vehicle to come 
into contact with a leading vehicle. When velocity V a is higher 
than velocity the relative velocity V r is negative. In this case, 
time to contact TTC Is negative. 

25 At box 226, the processor determines a period of time r 

p , which indicates how many seconds to come prior to a future 
environment to be predicted, in response to the time to contact 
TTC. The period of time r p is expressed as, 

30 r p = f(l/TTC) Eq. 9 

This function involves, as. a variable, the reciprocal of TTC. The 
fully drawn curve in Figure 31 illustrates this function. As shown 
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in Figure 31, the larger the reciprocal of TTC (1/TTC), the 
smaller the period of time r p is. 

The period of time r p corresponds to the magnitude of risk 
that is felt or perceived by the vehicle operator. In the case 
5 where time to contact TTC is negative or long so that the first 
extent (1/TTC) is small, the subject vehicle 70D separates 
sufficiently from the leading vehicle. The current environment in 
the field around vehicle 700 does not require urgent attention by 
the operator. In such environment, the operator can afford to 

10 predict future environment long time to come to account for all 
available potential risk. Thus, the period of time t p is long. In 
the case where time to contact TTC is short so that the first 
extent (1/TTC) is large, the subject vehicle 70D has approached 
to the leading vehicle. The current environment in the field 

is around vehicle 70D indicates the presence of risk in immediate 
future. In such environment, the operator will pay attention to 
the leading vehicle and cannot afford to predict future 
environment long time to come to account for all available 
potential risk. Thus, the period of time v p is short. 

20 With continuing reference to the flow diagram in Figure 30, 

after determining the period of time t p at box 226, the logic goes 
to box 228. At box 228, the processor inputs the acquired data 
at communication tool 208. What is read at box 228 includes; 

1. Operating state of a leading vehicle (such as, velocity 
25 V a and acceleration/deceleration) that is obtained 

through intervehicfe communication; and 

2. Traffic condition and road condition in front of the 
subject vehicle, which are obtained through 
infrastructure communication. 

30 Th use of intervehicfe communication is limited to the case 
where the leading vehicle has intervehicfe communication tool. 

At the next box 230, the processor inputs the acquired 
image data at front camera 76. The image data covering the 
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front view of vehicle 70D is subject to image and pattern 
recognition for he processor to determine the presence of a 
leading vehicle and whether its rear brake lamps are turned "on" 
or "off". The information as to the state of the brake lamps is 
s used later for predicting the magnitude of deceleration of the 
leading vehicle. 

At box 260, the processor predicts deceleration XG a of the 
leading vehicle. If the leading vehicle is capable of performing 
intervehicle communication, the processor inputs the 

10 information on deceleration XG a of the leading vehicle out of the 
acquired data through the intervehicle communication. If the 
leading vehicle is not capable of performing the intervehicle 
communication, the processor predicts the deceleration XG a in 
the following manner. 

is First, the processor determines, out of the acquired image 

data from the front camera 76, whether or not the brake lamps 
of the leading vehicle is turned "on". If this is the case, the 
processor measures, using a timer, for example, the elapse of 
time when the brake lamps are turned "on" Using the measure 

20 of the elapse of time and the variation of velocity V a of the 
leading vehicle, the processor predicts the deceleration XG a . For 
example, if the measure of the elapse of time is less than 0.5 
seconds, the processor sets a predetermined value as a virtual 
deceleration value of the leading vehicle. If the measure of the 

25 elapse of time is not less than 0.5 seconds, the processor 
determines a predicted value by calculation out of the rate of 
variation of past values of velocity V 9 of the leading vehicle. 

The predicted deceleration XG a of the leading vehicle is 
corrected accounting for the homogeneous degree of traffic flow. 

30 The homogeneous degree of traffic flow is predicted based on 
the traffic congestion of the road, which may be obtained, at box 
228, out of the acquired data from the communication tool 208. 
The logic is such that the homogeneous degree of traffic flow is 
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w/?en tfte traffic is congested and the variation of vehicle 
velocity V a is small. The processor corrects the predicted 
deceleration XG a in response to the predicted homogeneous 
degree of traffic flow. For example, the processor corrects the 
5 predicted deceleration XC a such that the predicted deceleration 
XG g decreases as the predicted homogeneous degree of traffic 
flow increases. This is because a vehicle will not be accelerated 
or decelerated greatly when the road is congested and the 
homogeneous degree of traffic flow is high. 

10 At the next block 234, the processor predicts future value 

of the first extent (TTC) that would occur r p after, using the 
current value of the first extent (TTC) obtained at box 224, the 
period of time r p determined at box 226, and the deceleration 
XG a predicted at box 232. First the processor calculates 

is velocity of the subject vehicle 70D, velocity of the leading 
vehicle, and the vehicle separation between the vehicles. The 
processor calculates deceleration XG f ofthe subject vehicle 70D 
based on the current velocity v f . When it is assumed that 
deceleration XGf and XG a remains unaltered during the period of 

20 time r p , future values of Vf, V a and D are expressed as, 

Vf(to+ r p ) = Vf(t 0 ) + XG f X r p Eq. 10 

V a (t 0 + r p ) = V a (t 0 ) + XG 9 x r p Eq. 11 

D(t 0 + r p ) = D(t 0 ) -V r xr p + (1/2) X (XG a - XG f ) x r p 2 Eq. 12 

25 

Using the results given by calculations of the above equations 
10-12, the processor predicts the future TTC, which is expressed 
as, 

30 TTC(t 0 + r p ) = D(t 0 + r p )/{ Vf(to+ r p ) - V a (t 0 + r P )> Eq. 13 
where, t D is the current moment 
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At box 236, the processor predicts future environment by 
ca leu fa ting RP, which is expressed as, 

RP = b/ TTC(t 0 + r p ) Eq. 14 

At box 238, the processor uses the RP to determine an 
accelerator pedal reaction command AF, which is expressed by 
the equation 7. 

At the next box 240, the processor outputs the command 
AFand applies it to the accelerator pedal reaction characteristic 
modulator 144 (see Figure 28). 

According to the fourth implementation, the period of time 
z p is determined as a function of time to contact TTC. Time to 
contact TTC is determined based on V f , V a , V r and D. Using the 
information from the acquired data by front camera 76 and 
communication tool 208, the deceleration of a leading vehicle 
XG a is predicted. Using this deceleration XG a , a future value of 
TTC that would occur r p after is predicted. Risk perceived PR is 
determined as a function of the reciprocal of this future value of 
TTC. Finally, the RP is used to determine accelerator pedal 
reaction command aF, 

According to the fourth implementation, a future value of 
extent to which vehicle 70D might approach to a leading vehicle 
is predicted from a current value of the extent. This future value 
of the extent corresponds to the real risk perceived or felt by the 
operator. In this implementation, RP (risk perceived) is 
indicative of this future value. The RP is predicted as a quantity 
indicative of future environment in a field around vehicle 70D. 
Accounting for the predicted RP, the accelerator pedal reaction 
control is performed. 

The period of time r p is used for prediction of a future value 
ofRP. The period of time r p is variable with the extent to which 
vehicle 70D has approached to the leading vehicle. The faster 
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the vehicle approaches to the leading vehicle, the more 
immediate future is selected for the prediction. This has 
provided an accelerator pedal reaction control that corresponds 
to the magnitude of risk actually perceived or felt by operator. 
5 Like the previously described third implementation, the operator 
is continuously kept informed of how fast the vehicle Is 
approaching to the leading vehicle through the magnitude of 
reaction force felt during manipulation of accelerator pedal. 

As different from the third implementation, the 

10 deceleration XG a of the leading vehicle, which is determined on 
the acquired data at front camera 76 and/or communication tool 
208, is used. The use of deceleration XG a has enhanced 
prediction of future environment in a manner more closely 
corresponding to the magnitude of risk actually perceived by 

is operator. 

In the fourth implementation, a current value and a future 
value of time to contact TTC are provided by calculation of 
equations involving, as variables, such physical quantities as 
vehicle speed and vehicle separation. This has lead to the 

20 minimal number of new components needed for installing a 
driving assist system in a vehicle. Using communication tool 208 
to receive information on deceleration XG a for use in the 
predication enhances the accuracy of a future value of TTC. 

In the fourth implementation, both front camera 76 and 

25 communication tool 208 are used to determine deceleration XG a . 
The present invention is not limited to this example. Another 
example is the use of front camera 76 or communication tool 
208. Further example is the use of information provided by laser 
radar 74 or source of vehicle speed 80. A future value of TTC as 

30 determined using such deceleration XG a as predicted is 
satisfactorily reliable. 

In the fourth implementation, laser radar 74 is used. Such 
laser radar may be replaced by millimeter radar or radar of other 
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type. 

/7/t/7 Implementation of the Invention 
The fifth implementation is substantially the same as the 
third implementation except the provision of producing a 
5 selected one of different alert categories to be informed of via an 
appropriate alarm. Thus, the same technique is used in the third 
and fifth implementations in determining TTC and THW. 

The flow diagram of Figure 32 illustrates a control routine 
of a first embodiment of the fifth implementation. The control 
io routine, generally indicated at 250, is executed at regular time 
intervals of 50 milliseconds. 

The control routine 250 is substantially the same as the 
control routine 180 illustrated in Figure 24 in that boxes 252, 
254 and 256 correspond to boxes 182, 184 and 186, 
is respectively. A description on the boxes 252, 254 and 256 is 
hereby omitted for brevity. The following paragraph provides a 
description on two boxes 258 and 260 that follow box 256. 

In Figure 32, at box 258, the processor of controller 82C 
(see Figure 20) selects one of alert categories in response to the 
20 RP. Various alert categories are stored for different ranges in 
magnitude of RP. For example, alert category elevates as RP 
increases, in magnitude, into the adjacent higher range from a 
current range. Each alert category may contain the repletion 
rate of alarm such that the repetition rate of alarm elevates with 
25 elevation of alert category. At output box 260, the processor 
outputs the selected alert category to the alarm. 

Setting alert category in response to the RP as described 
above will keep operator informed of the magnitude of risk when 
vehicle has approached to a leading vehicle and/or when 
30 influence due to a predicted future change in environment is 
great 

In this embodiment, the RP expressed by the equation 6 
has been used. Alternatively, RP expressed by the equation 8 or 
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14 may be used. An example using the equation 14 will be 
explained below. 

The flow diagram of Figure 33 illustrates a control routine 
of a second embodiment of the fifth implementation. The 
s control routine, generally indicated at 270, is executed at 
regular time intervals of 50 milliseconds. 

The control routine 270 is substantially the same as the 
control routine 220 illustrated in Figure 30 in that boxes 272, 
274, 276, 278, 280, 282, 284 and 286 correspond to boxes 222, 
xo 224, 226, 228, 230, 232, 234 and 236,. respectively. A 
description on the boxes 272, 274, 276, 278, 280, 282, 284 and 
286 is hereby omitted for brevity. The following paragraph 
provides a description on two boxes 288 and 290 that follow box 
286. 

is In Figure 33, at box 288, the processor of controller 82D 

(see Figure 28) selects one of alert categories in response to the 
RP that has been determined at box 286. Various alert 
categories are stored for different ranges in magnitude of RP. 
For example, alert- category elevates as RP increases, in 

20 magnitude, into the adjacent higher range from a current range. 
Each alert category may contain the repletion rate of alarm such 
that the repetition rate of alarm elevates with elevation of alert 
category. At output box 290, the processor outputs the selected 
alert category to the alarm. 

25 Setting alert category in response to the RP as described 

above will keep operator informed of the magnitude of risk when 
vehicle has approached to a leading vehicle and/or when 
influence due to a predicted future change in environment is 
great. 

30 In the embodiments of the fifth implementation, the 

repetition rate of alarm has been varied for different alert 
categories. The invention is not limited to the repetition rate of 
alarm. Another example is to vary the volume and/or tone of 
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alarm for different alert categories. Other example is to use 
various kinds of voice for different alert categories. 

Sixth Implementation of the Invention 

The sixth implementation is substantially the same as the 
third, fourth and fifth implementations in the use of equation 6 in 
determining the RP. However, according to the sixth 
implementation, the use of equation 6 is limited to provide a new 
RP, which does not keep operator from operating an automobile 
70C (see Figures 20-24) to approach to a leading vehicle when 
the vehicle separation D is sufficiently long. 

The fully drawn curve in Figure 34 illustrates the varying of 
RP expressed by the equation 6 with time t when the vehicle 
starts, at zero point (t = 0), approaching to the leading vehicle at 
the rate of a very low approaching speed of 10 km/hour. For 
ease of understanding, it is assumed that, during the first half 
from the zero point (t = 0) to a threshold moment (t - ta), the 
vehicle separation D between the vehicles is sufficiently long and 
the risk actually perceived or felt by the operator falls in a low 
risk level range. The second half begins immediately after the 
threshold moment (t = ta). During the second half, the vehicle 
separation D becomes shorter and shorter and the risk actually 
perceived by the operator grows with respect to time. 

During the first half, paying much attention mainly to THW, 
the operator manipulates an accelerator pedal. During this first 
half, the RP expressed by the equation 6 and felt by the operator 
through the accelerator pedal grows in response to the 
reciprocal of THW, which the operator pays attention to. The 
risk actually perceived by the operator does not grow in 
response to the reciprocal of THW, however. Specifically, during 
the first half, the RP expressed by the equation 6 grows at the 
rate less than the rate at which the risk actually perceived by 
operator grows. In Figure 34, the dotted line illustrates the 
varying of the risk actually perceived by operator during the first 
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half and the subsequent transient to the second half. 

According to the sixth implementation, a new risk 
perceived RPi is used to compensate for an insufficiency 
provided by the RP expressed by the equation 6. To avoid 
5 confusion, the RP expressed by the equation 6 is represented by 
a current risk perceived RP 0 . 

Thus, the RPo is expressed as, 

RP 0 = rp = a(l/THW) + b(l/TTC) Eq. 15 

10 

where b and a (b > a) are parameters weighting the first extent 
(l/TTC) and the second extent (1/THW), respectively, such that 
the second extent (1/THW) is less weighted than the first extent 
(l/TTC) is. The values of b and a are optimized after accounting 
is for a statistics of values of THW and TTC collected in the traffic 
scene including leading and trailing vehicles. In this 
implementation, b = 8 and a = 1. 

The RPi = (c- THW) + (b/TTC) Eq. 16 

20 

where b and c are parameters. In this implementation, b = 8 
and c = 2.5. 

In this implementation, the threshold of THW is 0.5 second 
25 (THW = 0.5). When THW > 0.5, the RP 2 is used. When THW ^ 
0.5, the RP 0 is used. The graph in Figure 35 illustrated this 
"hybrid" use of two RP 0 and RP 2 . In Figure 35, the one-dot chain 
line illustrates the variation of RPj against THW and the fully 
drawn curve illustrates the variation ofRP 0 against THW. In this 
30 implementation, the RP 2 is used when THW > 0.5, and the RP 0 is 
used when THW ^ 0.5. 

Seventh Implementation of the Invention 
The seventh implementation is substantially the same as 
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ffie s/xt/) implementation. In this seventh implementation, 
another RP2 is used. The RPz is expressed as, 

RPz = d (c- THW) + (a/THW) + (b/TTC) Eq. 17 

5 

where a, b, c and d are parameters. In this implementation, b = 
8, and c - 2.5. 

d and a are determined as follows: 
when THW S 0.5: d = 0, a = 1 
10 when THW > 0.5: d = 1, a = 0. 

Apparently, the equation 17 provides the same performance as 
the equations 15 and 16 provide. 

Referring to Figure 36, an improved version using the 

is equation 17 is described. According to this improved version, 
the parameters a and d used in the equation 17 are varied 
continuously in the illustrated pattern in Figure 36 to provide a 
smooth transition between two discrete RP 0 and RP 2 that exist in 
the equation 17. In Figure 36, the one-dot chain line illustrates 

20 the variation of parameter a with respect to THW, and the fully 
drawn line illustrates the variation of the parameter b with 
respect to THW. The fully drawn curve in Figure 37 illustrates 
the smooth varying of RP S when the parameters a and b are 
varied as illustrated in Figure 36. 

25 Eighth Implementation of the Invention 

Figure 38 provides hardware and Figure 39 illustrates how 
the hardware is arranged on an automobile 70E. 

In Figure 38, the reference numeral 40E generally 
indicates a driving assist system. Driving assist system 40E is a 

30 hybrid including an on board data acquisition system (DAS) 72 
and a steering reaction modulation actuator 84, which are 
substantially the same as their counterparts in the previously 
described driving assist system 40 A (Figure 3), and an 
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accelerator reaction modulation actuator 140 and a brake 
reaction modulation actuator 142, which are substantially the 
same as their counterparts in the previously described driving 
assist system 403 (Figure 5). 

In Figure 36, driving assist system 40E includes a 
microprocessor-based controller 82E. 

Controller 82 E recognizes the state of obstacles within 
environment in a field around vehicle 70E by determining 
velocity V f of vehicle 70E, relative position and relative velocity 
between vehicle 70E and each of other vehicles within the 
environment, and relative position between the adjacent lane 
marking and each of obstacles within the environment. Based 
on the recognized obstacle state, controller 82E determines risk 
which each of the recognized obstacles would cause the operator 
to perceive. Controller 82E divides each of the risks into a 
longitudinal risk component and a lateral risk component, and 
sums all of the longitudinal risk components to give a total 
longitudinal risk and all of the lateral risk components to give a 
total lateral risk. Based on the total longitudinal and lateral risks, 
controller 82E determines longitudinal commands and a lateral 
command. 

Controller 82E outputs the longitudinal commands for 
application to an accelerator pedal reaction characteristic 
modulator 144 and a brake pedal reaction characteristic 
modulator 148, respectively. In response to the applied 
longitudinal commands, the accelerator pedal reaction 
characteristic modulator 144 and the brake pedal reaction 
characteristic modulator 148 control servo motor 146 and brake 
booster ISO, thereby modulating the accelerator pedal and 
brake pedal reaction force characteristics. Modulating the 
accelerator pedal and brake pedal reaction characteristics 
prompt the vehicle operator to manipulating an accelerator 
pedal 152 and a brake pedal 154 to appropriate positions, 
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respectively. 

Controller 82E outputs the lateral command for application 
to a steering reaction characteristic modulator 86. In response 
to the applied lateral command, the steering reaction 
5 characteristic modulator 86 controls servo motor 88. Modulating 
the steering reaction characteristic prompts the vehicle operator 
to manipulating a steering wheel to an appropriate angular 
position. 

The flow diagram in Figure 40 illustrates a control routine 

io of the eighth implementation of the present invention. The 
control routine, generally indicated at 310, is executed at 
regular time intervals of 50 milliseconds. 

At input box 312, the processor of controller 82 E inputs 
acquired data by DAS 72. Specifically, the processor inputs 

is vehicle velocity V f , an angular location of a leading vehicle and a 
vehicle separation D between the subject vehicle 70E and the 
leading vehicle. The processor also inputs relative position of 
the vehicle 70E to the adjacent lane marking out of the image 
data from the front camera 76. The relative position of the 

20 vehicle 70E to the adjacent lane marking includes a lateral 
position and an angular position of the vehicle 7QE with respect 
to the lane marking. Out of the image data from the front 
camera 76, the processor further inputs the shape of the lane 
marking. Out of the image data from the front camera 76, the 

25 processor still further inputs an angular location of the leading 
vehicle and a vehicle separation D between the vehicle 70E and 
the leading vehicle. Out of the image data from RRS and RLS 
cameras 78R and 78L, the processor inputs an angular location 
of each of vehicles existing in the adjacent lane(s) and a vehicle 

30 separation between the vehicle 70E and each of the vehicles. 
The processor also input velocity of the vehicle 70E from the 
source of vehicle speed 80. Besides, the processor recognizes 
the presence of and kinds of obstacles around the vehicle 70E 
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ouf of the image data from the front camera 76 and the image 
data from RRS and RLS cameras 78R and 78L. Recognizing the 
kinds of obstacles includes pattern recognition to identify them 
whether they are a four wheel vehicle or a two wheel vehicle or 
5 a pedestrian. 

At box 314, the processor determines current 
environment in the field around the vehicle 70E in terms of 
position of each of the obstacles/vehicles relative to the vehicle 
70E, direction of movement of each of the obstacles relative to 

10 the vehicle 70E, and speed at which each of the obstacles 
relative to the vehicle 70E. Specifically, the processor 
determines current values of position, direction and speed of 
each of the obstacles/vehicles from the past stored values 
thereof that were determined in the previous cycles and the 

is input data at box 312. 

At box 316, using the data on current environment 
determined at box 314, the processor calculates time to collision 
TTC between the vehicle 70E and each of the obstacles. TTC k 
indicates TTC between the vehicle 70E and an obstacle k. TTCk 

20 is expressed as, 

TTCk = — ~~- Eq. 18 

where 

Die = Separation (or distance) between the vehicle 70E and an 
25 obstacle k; 

V rk = Relative velocity between the vehicle 70E and the obstacle 
k; 

o(D k ) - Variance of separation; and 
0(Vrk) = Variance of relative velocity. 



30 



The following several paragraphs provide a description on 
the variances a (D*) and a(Vrk). 
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An error in data might grow should if a sensor or a camera 
be subject to uncertainty or unforeseeable event Accounting 
for how much such an error might be, variance of separation o 
(D k ) and variance of relative velocity a(V rk ) are determined 
5 depending upon what an obstacle k is and which sensor or 
camera is used to recognize the obstacle k. 

For example, the laser radar 74 is superior to a CCD 
camera, which is used as the front camera 76 and RRS and RLS 
cameras 78R and 78L, in its capability of providing accurate 

10 measure in separation or distance between two vehicles 
irrespective of how far they are separated. The curves in Figure 
41 illustrate the variances of separation cr(Dk) set for laser radar 
74 and CCD camera, respectively. The CCD camera is used for 
front camera 76 and RRS and RLS cameras 78R and 78L. As 

is shown in Figure 41, when laser radar 74 is used to measure a 
separation Dk between the vehicle 70E and an obstacle k in the 
form of an automobile, variance cx(Dk) is kept constant over any 
different values in the separation to be measured. When the 
CCD camera is used to measure the separation D k , variance a 

20 (Dk) increases exponentially. When the separation D k is less 
than a predetermined value, the CCD camera provides better 
accuracy than laser radar 74 does, and variance o(D k ) for CCD 
camera is less than that for laser radar 74. 

The curves in Figure 42 illustrate the variances of relative 

25 velocity o (V rk ) set for laser radar 74 and CCD camera, 
respectively. As shown in Figure 42, when laser radar 74 is used 
to measure relative velocity V rk between the vehicle 70E and an 
obstacle k in the form of an automobile, variance a (Vrk) 
increases linearly as the relative velocity V rk increases. When 

30 the CCD camera is used to measure the relative velocity V rk , 
variance cr(V rk ) increases exponentially. 

Using pattern recognition technique, image data from 
front camera 76 and RRS and RLS cameras 78R and 78L are 
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analyzed by the processor to determine what an obstacle k is. A 
family of curves in Figure 43 illustrates the variance o(D k ) set for 
CCD camera. A family of curves in Figure 44 illustrates the 
variance a(V rk ) set for CCD camera. A four wheel vehicle, a two 
5 wheel vehicle, a pedestrian and lane markings were selected 
and used as an obstacle k for measurement by the CCD camera. 

When the CCD camera is used to measure the separation 
Die, the bigger the obstacle k, the higher the accuracy of 
measurement is. Accordingly, as shown in Figure 43, the 

10 variance o (D^) for measuring separation from a four wheel 
vehicle is less than that for measuring separation from two 
wheel vehicle or pedestrian. 

When the CCD camera is used to measure the relative 
velocity V fk , the variance a (V rk ) is determined depending on the 

is magnitude of relative velocity which may be assumed for the 
kinds of obstacle k, respectively. As shown in Figure 44, the 
higher the assumed relative velocity, the larger the variance a 
(V rk ) is. Comparing relative velocity assumed for four wheel 
vehicle to that assumed for two wheel vehicle or that assumed 

20 for pedestrian, the former is higher than the latter. Accordingly, 
the variance cr(Vrk) for four wheel vehicle is set greater than that 
for two wheel vehicle or that for pedestrian. 

In Figure 43, the variance o (D k ) for lane marking is set 
less than those for four wheel vehicle, two wheel vehicle and 

25 pedestrian. In Figure 44, the variance cf(V rk ) for lane marking is 
set less than those for four wheel vehicle, two wheel vehicle and 
pedestrian. 

With reference again to the flow diagram in Figure 40, 
after calculating TTC k at box 316, the logic goes to box 318. At 
30 318, the processor uses TTCk to calculate risk perceived RP k , 
which an obstacle k would cause operator to perceive. RP k is 
expressed as, 
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RP k = _L_ xW Eq.I9 
7TC* 



= Weighting component applied to the obstacle k. 
s Various values are set as weighting component for different 
obstacles in kind. For example, when a four wheel, a two wheel 
vehicle or a pedestrian is an obstacle k f the operator would be 
influenced most upon approaching to such obstacle k. Thus, in 
this implementation, w k = 1, when obstacle k is one of a four 

10 wheel, a two wheel vehicle and a pedestrian. When lane 
marking is an obstacle k, the operator would be influenced less 
upon approaching to the Jane marking. Thus, in this 
implementation, W k = about 0.5, when obstacle k is lane 
marking. Operator would be influenced less upon approaching 

is to lane marking beyond which lies a railroad or a wall or a fence 
than one would be influenced upon approaching to lane marking 
beyond lies the adjacent lane. Thus, different values may be set 
as Wk depending upon where the lane marking exists. 

20 Lane markings are distributed within a range of angular 

locations ahead of the vehicle 70E. Risk with regard to lane 
markings is determined by integrating RPk over the whole range 
of angular locations. Risk perceived RP lan e is expressed as, 

25 RP ld ne = f(— }—*W>J)dL Eq. 20 

where 

W iane = Weighting component 

30 With continuing reference to Figure 40, at box 320, the 

processor sums longitudinal components of individual risk 
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perceived RPk including risk with regard to lane markings RP/ an e 
to provide a total longitudinal risk RP L ongituamah The total 
longitudinal risk RP Longitudinal is expressed as, 

5 RP Longitudinal = ^J*P*COSft Eq. 21 

where, 

Ok- Angular location of an obstacle k as viewed from the 
vehicle 70E 0 k = 0 when the obstacle k is located in front of the 
vehicle. d k = 180 when the obstacle k is located behind the 
10 vehicle. 

At the next block 322, the processor sums lateral 
components of individual risk perceived RP k including risk with 
regard to lane markings RPiane to provide a total lateral risk 
is RPLaterah The total longitudinal risk RPuterai is expressed as, 

RP Lateral = ^RP>s\nd> Eq. 22 

t 

At box 324, the processor determines longitudinal 
20 commands by referring to the relationship shown in Figure 45 
and the relationship shown in Figure 46 using the total 
longitudinal risk RP Longitudinal- One of the longitudinal commands, 
namely, an accelerator pedal reaction command F*, is provided 
to modulate an accelerator pedal reaction characteristic. The 
25 other command, namely, a brake pedal reaction command F B , is 
provided to modulate a brake pedal reaction characteristic. 
They provide such a relationship that, when the total 
longitudinal risk RPtongitudinai increases, the accelerator pedal 
reaction command F A increases, prompting operator to re/easing 
30 the accelerator pedal 152, while the brake pedal reaction force 
Fb decreases, prompting .the operator to depressing the brake 
pedal 154. 
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7/7e ft///y drawn line in Figure 45 illustrates the relationship 
between the accelerator pedal reaction command F A and the 
total longitudinal risk RP Longitudinal As will be readily seen from 
Figure 45, the command F A remains zero when RP Longitudinal is less 
5 than a first threshold RP 0 . When RP Longitudinal exceeds the first 
threshold RP 0 but it is still less than a second threshold RP m ax 
that is greater than the first threshold RP 0 , the command F A is 
proportional to RP Longitudinal The command F A takes the 
maximum when RP L ongitudina/ exceeds the second threshold RPmax- 

10 The fully drawn line in Figure 46 illustrates the relationship 

between the brake pedal reaction command F B and the total 
longitudinal risk RP Longitudinal- When RP Longitudinal is greater than 
the second threshold RPmax, the greater longitudinal risk 
RP Longitudinal, the less the command F B is. When RP Longitudinal is 

is greater than a third threshold RP Z that is greater than the second 
threshold RPmax, the command F B is at the minimum F Bm i n , 
holding the brake pedal reaction force to the minimum. When 
RPLongitudinai is not greater than the second threshold RP m ax, the 
command F B is zero, leaving the brake pedal reaction 

20 characteristic unaltered. 

With reference to Figures 45 and 46, when the longitudinal 
risk RP L ongitudinai is not greater than the threshold RPmax, the 
accelerator pedal reaction characteristic is altered, letting 
operator know the magnitude of longitudinal risk RPLongitudinai by 

2s means of a change in reaction force of the accelerator pedal. 
When the longitudinal risk RPLongitudinai exceeds the threshold 
RPmax, the accelerator pedal reaction force is maximized, 
prompting the operator to releasing the accelerator pedal. At 
the same time, the brake pedal reaction force is reduced, 

30 allowing the brake pedal to be depressed easily when the 
operator steps on the brake pedal. 

In Figure 40, at box 326, the processor determines a 
lateral command by referring to the relationship shown in Figure 
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47 using the total lateral risk RPLaterai- The lateral command, 
namely, a steering reaction command F s , is provided to 
modulate a steering reaction characteristic. It provides such a 
relationship that, when the total lateral risk RPtateraii increases, 
5 the steering reaction command F s increases in a manner to 
prompt operator to steering the vehicle to reduce the risk 
RPiaterat- The fully drawn line in Figure 47 provides the 
relationship between the total lateral risk RPLaterai and the 
steering reaction command F s . In Figure 47, the total lateral risk 

10 RPLaterai is positive when risk on the right hand side exceeds that 
on the left hand side, but it is negative when risk on the left hand 
side exceeds than on the right hand side. 

As shown in Figure 47, the steering reaction command F s 
ranges from 0 to a maximum right-turn reaction F S max when the 

is total lateral risk RPLaterai is greater , than 0 but less than a 
right-turn threshold RP max . When the total lateral risk RPuterai is 
in the vicinity of zero, the command F s is zero, leaving steering 
reaction characteristic unaltered. Subsequently, when the total 
lateral risk RPtaterai increases, the command F s increases with the 

20 increasing total lateral risk RPLaterai. When the total lateral risk 
RPLaterai is equal to or greater than the right-turn threshold RP max , 
the command F s stays at the maximum right-turn reaction Fs max . 
As the command F s increases from zero, the reaction force to 
operator right turn steering effort increases from a level of the 

25 usual steering reaction characteristic. 

With continuing reference to Figure 47, the steering 
reaction command Fs ranges from 0 to a maximum left-turn 
reaction -F Smax when the total lateral risk RP La tersi is less than 0 
but greater than a left-turn threshold -RP mBx . When the total 

30 lateral risk RPtaterai is in the vicinity of zero, the command F s is 
zero, leaving steering reaction characteristic unaltered. 
Subsequently, when the total lateral risk RP La ten,i decreases, the 
command F s decreases with the decreasing total lateral risk 
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RPLaterah When the total lateral risk RPuxerat is equal to or less 
than the left-turn threshold -RP max , the command Fs stays at the 
maximum left-turn reaction -F Sm ax. As the command F s 
decreases from zero, the reaction force to operator left turn 
5 steering effort increases from a level of the usual steering 
reaction characteristic. 

With reference again to Figure 40, at output box 328, the 
processor outputs the commands determined at boxes 324 and 
326 using the illustrated relationships in Figures 45, 46 and 47. 

10 These commands include the accelerator pedal reaction 
command Fa, the brake pedal reaction command F B , and the 
steering reaction command F s for application to the accelerator 
pedal reaction characteristic modulator 144, brake pedal 
reaction characteristic modulator 148 and steering reaction 

15 characteristic modulator 86 (see Figure 38), respectively. 

The preceding description on the flow diagram in Figure 40 
clearly indicates that the controller 82E accounts for longitudinal 
and lateral risk components calculated based on individual risk 
derived from each of obstacles within a field around the vehicle 

20 70E. In response to the longitudinal and lateral risk components, 
the controller 82E allocates commands Fa, F b and F s for 
prompting operator to operating the vehicle 70E longitudinally 
and laterally in an appropriate manner to cope with such risk 
components. This may be readily confirmed by referring to the 

25 description on boxes 320, 322, 324 and 326. In the eighth 
implementation of the invention, this allocating function is 
implemented in terms of software (instructions) component. 
However, the invention is not limited to this. The allocating 
function may be implemented in terms of software and hardware 

30 components or hardware component. 

In the eighth implementation, the controller 84E 
determines the magnitudes of accelerator pedal reaction 
command F A , brake pedal reaction command F B , and steering 
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reaction command F s based on the total longitudinal risk 
RPLongitudinat and total lateral risk RP^terah In this implementation, 
the magnitudes of such commands are determined using the 
relationships illustrated in Figures 45, 46 and 47. These 
5 illustrated relationships show only one example of an operator 
response plan that prompts operator to operating the vehicle 
70E in a manner to negotiate with the longitudinal and lateral 
risks RPiongitudinat and RPiaterai- In this example, the accelerator 
pedal reaction characteristic, brake pedal reaction characteristic 

10 and steering reaction characteristic are modulated or altered to 
prompt the operator to operating the vehicle 70E in a direction 
toward where the longitudinal and lateral risks RPtongitudinai and 
RPiaterai are lowered. 

With reference to Figures 38 to 47, the eighth 

is implementation of the present invention has been described. 
This implementation provides features as follows. 

(1) As information on environment in a field around the 
vehicle 70E, the controller 82E determines the presence of all of 
obstacles, including a leading vehicle, a trailing vehicle, a vehicle 

20 in the adjacent lane, a pedestrian, and lane markings. The 
controller 82 E calculates, for each of the obstacles, a risk RP k . 
Accounting for the risks RPk from all directions of the vehicle 40E, 
the controller 82E allocates commands Fa, F B/ and F s for 
prompting the vehicle operator to operating the vehicle 40E 

25 longitudinally and laterally in a direction appropriate to cope with 
the available risks RP k . It will be appreciated that this 
implementation provides an appropriate assist to the vehicle 
operator. 

(2) The controller 82E provides an appropriate assist to 
30 manual effort to operate the accelerator pedal by modulating or 

altering accelerator pedal reaction in response to the 
determined command F A .. 

(3) For each of the obstacles, the controller 82E calculates 
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TTCk by dividing separation D k by relative velocity V rk , and 
determines risk RP k as a function of the reciprocal of TTC k - This 
risk RPk represents an extent how quickly the vehicle 70E has 
approached to the obstacle k. This implementation provides an 
5 appropriate assist to the vehicle operator to opening the vehicle 
70E longitudinally and laterally in response to the extent how 
quickly the vehicle 70E has approached to the obstacle k. 

(4) As is indicated by equation 18, TTC k = D '~° r(Z><) , 

K*+cr(Ki) 

determining TTCk involves variance of separation o (Dk) and 
io variance of relative velocity cr(V rk ). In use of laser radar 74, the 
variances o(Dk) anda(V r k) are subject to the illustrated varying 
in Figures 41 and 42. In use of CCD camera, the variances o(Dk) 
and a (V rk ) are subject to the illustrated varying in Figures 43 and 
44. Employing the variances a (Dk) anda(V r k) in determining 
is TTCk allows use of diverse radars and cameras in on board DAS 
72. 

(5) As is indicated by equation 19, RP k = — ^— *w> , 

TTCk 

determining RPk involves weighting component W k . The 
weighting component W k takes an appropriate value that has 

20 been selected out of various values predetermined for different 
kinds of obstacles and for different surroundings of lane marking. 
Employing the weighting component W k in determining RPk 
a/lows diverse obstacles to be processed in determining 
commands F A , F B and F s . 

25 (6) With respect to the vehicle 70E, angular locations of all 

of the obstacles cause the controller 82E to calculate longitudinal 
and lateral risk components and provide appropriate allocation 
of various commands Fa, F b and F s to the longitudinal and lateral 
risks. Accordingly, the controller 82E allocates commands F A , F B 

30 and F s to prompting operator to operating the vehicle 
longitudinally and laterally in accordance with the angular 
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locations of the obstacles. 

(7) The controller 82E provides an appropriate assist to 
manual effort to operate the steering wheel by modulating or 
altering steering reaction in response to the determined 

5 command F s . Further, the controller 82E provides an 
appropriate assist to manual effort to operate the brake pedal by 
modulating or altering brake pedal reaction in response to the 
determined command F a . 

(8) For each of the obstacles, the controller 82E calculates 
io risk RP k to calculate longitudinal and lateral risk components 

RP Longitudinal and RPt_aterBh In response to the longitudinal and 
lateral risk components, the controller 82E allocates commands 
Fa, F b and F s for prompting operator to operating the vehicle 70E 
longitudinally and laterally in an appropriate manner to cope 

is with such risk components. In this implementation, therefore, 
the driving system provides assist the operator in prompting to 
operating the vehicle longitudinally and laterally in response to 
distribution of individual risks RPk. In this implementation, the 
controller 82E determines RPk as a function of the reciprocal of 

20 TTCk. Therefore, the driving system assist the operator in 
prompting to operating the vehicle longitudinally and laterally in 
response to the extent how quickly the vehicle has approached 
to the obstacle k. 

In this implementation, the controller 82E uses the 

25 equation 19 to determine RP k as a physical quantity quantifying 
the extent how quickly the vehicle 70E has approached to an 
obstacle k. The invention Is not limited to the use of equation 1 9 
to quantify the extent how quickly the vehicle 70E has 
approached to an obstacle k. Any appropriate physical quantity 

30 may be used in quantifying the extent how quickly the vehicle 
70E has approached to an obstacle k. 

Ninth Implementation of the Invention 
The ninth implementation is substantially the same as the 
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eighth embodiment illustrated in Figures 38 to 47. 

Turning back to Figures 38 and 39, Figure 38 provides 
hardware of the ninth implementation and Figure 39 illustrates 
how the hardware is arranged on an automobile 70E according 
s to the ninth implementation. 

In the ninth implementation, a microprocessor- based 
controller 82E recognizes the state of obstacles within 
environment in a field around vehicle 70E by determining 
velocity V f of vehicle 70E, relative position and relative velocity 

io between vehicle 70E and each of other vehicles within the 
environment, and relative position between the adjacent lane 
marking and each of obstacles within the environment Based 
on the recognized obstacle state, controller 82E determines risk 
which each of the recognized obstacles would cause the operator 

is to perceive. Controller 82E predicts a change in risk RP k in 
response to a change in operator input. Controller 82E 
calculates a change in total longitudinal risk in response to a 
change in operator input for longitudinal operation of the vehicle 
70E and a change in total lateral risk in response to a change in 

20 operator input for lateral operation of the vehicle 70E. Based on 
the calculated changes in total longitudinal and lateral risks, 
controller 82E determines longitudinal and lateral commands. 

Controller 82E inputs the information on the current 
operator input for longitudinal operation of the vehicle by 

25 detecting positions of an accelerator pedal 152 and a brake 
pedal 154 (see Figure 39). Stroke sensors may be used to 
measure the positions of such pedals. The positions of such 
pedals may be determined from the state of a servo motor 146 
and the state of a brake booster 150. 

30 Controller 82 inputs the information on the current 

operator input for lateral operation of the vehicle by detecting 
manipulated angle of the steering wheel. 

Controller 82E outputs the longitudinal commands for 
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application to an accelerator pedal reaction characteristic 
modulator 144 and a brake pedal reaction characteristic 
modulator 148, respectively. In response to the applied 
longitudinal commands, the accelerator pedal reaction 
characteristic modulator 144 and the brake pedal reaction 
characteristic modulator 148 control servo motor 146 and brake 
booster 150, thereby modulating the accelerator pedal and 
brake pedal reaction force characteristics. Modulating the 
accelerator pedal and brake pedal reaction characteristics 
prompt the vehicle operator to manipulating the accelerator 
pedal 152 and the brake pedal 154 to appropriate positions, 
respectively. 

Controller 82E outputs the lateral command for application 
to a steering reaction characteristic modulator 86. In response 
to the applied lateral command, the steering reaction 
characteristic modulator 86 controls servo motor 88. Modulating 
the steering reaction characteristic prompts the vehicle operator 
to manipulating the steering wheel to an appropriate angular 
position. 

The flow diagram in Figure 48 illustrates a control routine 
of the ninth implementation of the present invention. The 
control routine, generally indicated at 340, is executed at 
regular time intervals of 50 milliseconds. 

The control routine 340 is substantially the same as the 
control routine 310 in Figure 40 In that boxes 342, 344, 346 and 
348 in Figure 48 are the same as the boxes 312, 314, 316 and 
318 in Figure 40, respectively. Accordingly, the detailed 
description on the boxes 342, 344, 346 and 348 is hereby 
omitted for brevity. 

In Figure 48, after the box 348, the control logic goes to 
box 350. 

At input box 350, .the processor inputs information on 
current operator input through the depressed angle e AB of 
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accelerator pedal 152 and/or brake pedal 154 and through the 
manipulated angle of steering wheel. When the operator 
manipulates the accelerator pedal 152, the processor uses the 
depressed angle of the accelerator pedal 152 as the depressed 
5 angle Gab. When the operator manipulates the brake pedal 154, 
the processor uses the depressed angle of the brake pedal 154 
as the depressed angle o A b- 

At the next box 352, the processor predicts a change in 
RP k in response to a change In operator input. Specifically, 

10 based on the current RPk determined at box 348 and the current 
operator input received at box 350, the processor predicts two 
new values ofRP k when there is a unit change from the current 
operator input in both directions. 

With reference to Figure 49, with regard to the operator 

is input for operating the vehicle in the longitudinal direction, the 
processor predicts an increase in vehicle velocity when there is 
an increase by a variation A AB from the current 
accelerator/brake pedal depressed angle & AB and it also 
predicts a decrease in vehicle velocity when there is a decrease 

20 by the variation AAB from the current accelerator/brake pedal 
depressed angle GAB. Based on the predicted increase and 
decrease in vehicle velocity, the processor calculates the new 
values of RP k using the equation 19. In other words, the 
processor predicts one new value of RP k after the vehicle velocity 

25 undergoes the predicted increase, and the other new value of 
RPk after the vehicle velocity undergoes the predicted decrease. 
Turning back to the equation 19, RP k is determined as a function 
of the reciprocal of TTC k that is determined as a function of 
relative velocity V*. From this explanation, it is clear that a 

30 change in vehicle velocity causes a change in vehicle velocity v rk 
which in turn causes a change in RP k . 

For ease of description, it is hereby assumed that the 
accelerator/brake pedal depressed angle OAB increases when 
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t/?e accelerator pedal 152 or brake pedal 154 is manipulated in a 
direction to cause an increase in RP k . For example, it is 
considered that there is an increase in accelerator/brake pedal 
0 AB when there is an increase from the current value of 
s depressed angle of the accelerator pedal 152 or there is a 
decrease from the current value of depressed angle of the brake 
pedal 154. 

With continuing reference to Figure 49, with regard to the 
operator input for operating the vehicle in the lateral direction, 

10 the processor predicts a change in heading direction of the 
vehicle when there is an increase by a variation AS from the 
current steering angle OS and it also predicts a change in 
heading direction of the vehicle when there is a decrease by the 
variation AS from the current steering angle 9S. Based on the 

is predicted changes in heading direction of the vehicle, the 
processor calculates new angular locations of each obstacle k 
and new values of RP k . If there is a change in heading direction, 
there is a change in separation between the vehicle and each 
obstacle and a change in angular location of the obstacle. 

20 Accordingly, a change in steering angle causes a change in RPk- 
In Figure 49, when the steering is in neutral position, OS is zero. 
9S is positive upon manipulating the steering wheel to the right, 
and it is negative upon manipulating the steering wheel to the 
left. 

25 The magnitudes of the variations A AB and A S are 

determined taking into account changes in pedal depressed 
angle and in steering angle per each processing time, which 
changes would normally occur during normal operation of the 
vehicle. In this implementation, the one processing time is, for 

30 example, from 0.1 to 0.2 seconds. The processing time differs 
depending on the kind of vehicle and on the type of processing to 
be completed. If the processing time ranges 0.1 to 0.2 seconds, 
AAB and AS are 10 mm and 0.5 degrees, respectively. The 
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variances are not limited to such fixed values. By using learning 
technique, the average of depressed angles of accelerator/brake 
pedal and the average of manipulated steering angles over one 
driving cycle may be set as the variances A AB and AS for the 
s next driving cycle. 

With reference again to the flow diagram in Figure 48, at 
box 354, the processor calculates a change in RP Longitudinal- Based 
on RPk determined at box 348 and its predicted new values 
determined at box 352, the processor determines the current 
10 total longitudinal risk RPuongitudimiCO), a new value of total 
longitudinal risk RPi_ongitudin<>i(+) for 0AB +AAB, and the other 
new value of total longitudinal risk RPLongitrudmaK-) for 0AB - AAB. 
The current total longitudinal risk RPLongitudinai(O) is expressed as, 

15 RPLongitudJnat(O) = £^cos& 23 

• 

Using the equation 21, the processor determines the new value 
of total longitudinal risk RPLongitudinat(+) for 0AB + AAB, and the 
other new value of total longitudinal risk RPLongitrudinai(-) ford A3 

20 -AAB. Figure 50 Illustrates the relationship between the current 
and two new values of total longitudinal risk. 

At the next box 356, the processor calculates a change in 
RPiaterai- Based on RPk determined at box 348 and its predicted 
new values determined at box 352, the processor determines 

25 the current total lateral risk RPu,terai(0), a new value of total 
lateral risk RP L etcrvi(+) for 0S +AS, and the other new value of 
total lateral risk RP La terai(~) for 05 -AS. The current total lateral 
risk RPtatentCO) is expressed as, 

30 RPLateralCO) m ^RP.sin& Eq. 24 

k 

Using the equation 22, the processor determines the new value 



llfllUCKMUl I USM 



-68- P02NM 045US 01-00643 

of total lateral risk RP La terat(+) for 0S +AS, and the other new 
value of total lateral risk RPiaterai(-) for&S -AS. Figure 51 
illustrates the relationship between the current and two new 
values of total lateral risk. 
5 At box 358, the processor determines longitudinal 

commands including a set of accelerator pedal reaction 
commands F A (0), F A (-) and F A (+), and a set of brake pedal 
reaction commands Fb(0), F b (-) and F B (+). The processor 
determines accelerator pedal reaction commands F A (0), F A (-) 
io and F A ( +) by referring to the relationship illustrated in Figure 52 

USing RPLongltudlnal(O), RPtongitudinaK-) and RP longitudinal +), 

respectively. The fully drawn line in Figure 52 is substantially 
the same as that in Figure 45. The processor determines brake 
pedal reaction commands F B (0), F B (-) and F B (+) by referring to 

is the relationship illustrated in Figure 53 using RP L ongitudinai(0), 
RPLongitudinai(-) and RPtongitudinaiC +), respectively. The fully drawn 
line in Figure 53 is substantially the same as that in Figure 46. 

At box 360, the processor determines a set of lateral 
commands F s (0), F s (-) and F s (+) by referring to the illustrated 

20 relationship in Figure 54 using RP Late rai(0), RP L aterai(-) and 
RPLaterai(+), respectively. The fully drawn line in Figure 54 is 
substantially the same as that in Figure 47. 

At output box 362, the processor outputs longitudinal and 
lateral commands determined at boxes 358 and 360. These 

2 s commands Include a set of accefera tor pedal reaction commands 
Fa(0), F a (~) and F A (+) for application to an accelerator pedal 
reaction characteristic modulator 1 44 (see Figure 38). They also 
Include a set of brake pedal reaction commands F B (0), Fb(-) and 
Fb(+) for application to a brake pedal reaction characteristic 

30 modulator 148 (see Figure 38). They further include a set of 
lateral commands F s (0), F s (-) and F s (+) for application to a 
steering reaction characteristic modulator 86 (see Figure 38). 
With reference to Figure 38, in response to the commands 
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a/7t/ /^f+j from the controller 82E, the accelerator 
pedal reaction characteristic modulator 144 controls a servo 
motor 146 to modulate or alter the accelerator pedal reaction 
characteristic as shown in Figure 55. In Figure 55, broken line 
s indicates the usual accelerator pedal reaction characteristic. At 
GAB, the accelerator pedal reaction is increased from the level 
on the usual reaction characteristic by F A (0). Upon manipulating 
the accelerator pedal from the current depressed angle toward 
0AB - AAB, the operator feels reaction force increased from the 

10 level on the usual reaction characteristic by F A (-J- Upon 
manipulating the accelerator pedal from the current depressed 
angle toward 6 AB + AAB, the operator feels reaction force 
increased from the level on the usual reaction characteristic by 
F A (+). From a change in reaction force, the operator can 

is recognize how the current risk will vary upon moving the 
accelerator pedal from the current position. 

In response to the commands F B (Q), F B (~) and F B (+), the 
brake pedal reaction characteristic modulator 148 controls a 
brake booster 150 to modulate or alter the brake pedal reaction 

20 characteristic as shown in Figure 56. In Figure 56, broken line 
indicates the usual brake pedal reaction characteristic. At GAB, 
the brake pedal reaction is as high as the level on the usual 
reaction characteristic because F B (0) = 0. Upon manipulating 
the brake pedal from the current depressed angle toward GAB - 

26 AAB, the operator feels the usual reaction force because F B (~) = 
0, Upon manipulating the brake pedal from the current 
depressed angle toward GAB +AAB, the operator feels reaction 
force decreased from the level on the usual reaction 
characteristic by F&(+). From a change in reaction force, the 

30 operator can recognize how the current risk will vary upon 
moving the brake pedal from the current position. 

In response to commands F s (0), F s (-) and F s (+), the 
steering reaction characteristic modulator 86 controls a servo 
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motor 88 to modulate or after the steering reaction 
characteristic as shown in Figure 57. In Figure 57, broken line 
indicates the usual steering reaction characteristic. At 0S, the 
steering reaction is increased from the level on the usual 
s reaction characteristic by Fs(0). Upon manipulating the steering 
wheel from the current angular position toward 0S - AS, the 
operator feels reaction force increased from the level on the 
usual reaction characteristic by Fs(-). Upon manipulating the 
steering wheel from the current angular position toward 05 + A 

10 S, the operator feels reaction force increased from the level on 
the usual reaction characteristic by F s (+). From a change in 
reaction force, the operator can recognize how the current risk 
will vary upon turning the steering from the current position. 

From the preceding description, it will now be understood 

is that the eighth and ninth implementations provide the same 
features. However, the ninth implementation provides a feature 
which the eighth implementation does not According to the 
ninth implementation, gradient of reaction force is created 
across the current position of operator input so as to prompt the 

20 operator to selecting a desired change from the current position 
of the operator input to reduce the risk. 

Tenth Implementation of the Invention 
The tenth implementation is substantially the same as the 
eighth embodiment illustrated in Figures 38 to 47. 

25 Turning back to Figures 38 and 39, Figure 38 provides 

hardware of the tenth implementation and Figure 39 illustrates 
how the hardware is arranged on an automobile 70E according 
to the tenth implementation. 

In the tenth implementation, a microprocessor-based 

30 controller 82E recognizes the state of obstacles within 
environment in a field around vehicle 70E by determining 
velocity V f of vehicle 70E, relative position and relative velocity 
between vehicle 70E and each of other vehicles within the 
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environment, and relative position between the adjacent lane 
marking and each of obstacles within the environment Based 
on the recognized obstacle state, controller 82E determines 
individual risk which each of the recognized obstacles would 
5 cause the operator to perceive. Controller 82E sums all of the 
individual risks to give a total risk. Controller 82E determines an 
angular location in which the total risk originates. In response 
to the determined angular position, controller 82E determines 
gains to be applied to a total longitudinal risk and to a total 
10 lateral risk. Based on the gains and the total longitudinal and 
lateral risks, controller 82E determines longitudinal and lateral 
commands. 

Controller 82E outputs the longitudinal commands for 
application to an accelerator pedal reaction characteristic 

is modulator 144 and a brake pedal reaction characteristic 
modulator 148, respectively. In response to the applied 
longitudinal commands, the accelerator pedal reaction 
characteristic modulator 144 and the brake pedal reaction 
characteristic modulator 148 control servo motor 146 and brake 

20 booster 150, thereby modulating the accelerator pedal and 
brake pedal reaction force characteristics. Modulating the 
accelerator pedal and brake pedal reaction characteristics 
prompt the vehicle operator to manipulating the accelerator 
pedal 152 and the brake pedal 154 to appropriate positions, 

25 respectively. 

Controller 82E outputs the lateral command for application 
to a steering reaction characteristic modulator 86. In response 
to the applied lateral command, the steering reaction 
characteristic modulator 86 controls servo motor 88. Modulating 

30 the steering reaction characteristic prompts the vehicle operator 
to manipulating the steering wheel to an appropriate angular 
position. 

The flow diagram in Figure 58 illustrates a control routine 
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of the tenth implementation of the present invention. The 
control routine, generally indicated at 370, is executed at 
regular time intervals of 50 milliseconds. 

The control routine 370 is substantially the same as the 
s control routine 31 0 in Figure 40 in that boxes 372, 374, 376, 378, 
380 and 382 in Figure 58 are the same as the boxes 312, 314, 
316, 318, 320 and 322 in Figure 40, respectively. Accordingly, 
the detailed description on the boxes 372, 374, 376, 378, 380 
and 382 is hereby omitted for brevity. 

10 In Figure 58, after the box 382, ^the control logic goes to 

box 384. At box 384, based on the total longitudinal risk 
RPtongitudinai determined at 380 and the total lateral risk RP La t e ra/, 
the processor calculates an angular location <t> HL in which a 
combined risk originates. The angular location <t> Ml is expressed 

is as, 

0ALL = tan*J¥^l- Eq , 25 

When the combined risk originates in an angular location ahead 

20 the vehicle, <t> AU = 0 degree. When it originates in an angular 

location behind the vehicle, 0 m = 180 degrees. When it 

originates in an angular location on the right hand or left hand 
side of the vehicle, \^\ = 90 degrees. 

At the next box 386, the processor determines gains 

25 G Longitudinal and G L aterai by referring to the illustrated curves in 
Figure 59 using the absolute value |^| at box 386. In Figure 59, 

the horizontal axis represents the absolute value of<t> MU while 

the vertical axis represents gain. The gain G Lo „g it udinai is 1 when 
the absolute value is less than a value 4>i that is less than 

30 90 degrees. This is the case where the combined risk originates 

ahead of the vehicle. The gain G Lo ngitudinai drops from 1 toward 
zero when the absolute value |<zW| exceeds the value 4> 2 . At 90 
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degrees, the gain G Lo ngitudinai is zero. This is the case where the 
combined risk originates form the side or from behind. 

With continuing reference to Figure 59, the gain G Lat erai is 1 
at about 90 degrees. This is the case where the combined risk 
5 originates on the lateral side of the vehicle. At 0 and 180 
degrees, the gate G L atrai drops down to zero. 

At box 388, the processor determines an accelerator pedal 
reaction command F A and a brake pedal reaction F B in the same 
manner as the processor did at box 324 in Figure 40. As 
10 different from box 324, the processor multiplies G Longitudinal with 
each of F A and Fb at box 388. 

At box 390, the processor determines a steering reaction 
command F s in the same manner as the processor did at box 326 
in Figure 40. As different from box 326, the processor multiplies 
is G La teraff with Fs at box 390. 

At box 392, the processor outputs the commands 
determined at boxes388and 390 for application to modulators 
144, 148 and 86 (see Figure 38). 

In the tenth implementation, gains GLongitudinai and GLaterai 
20 are determined for longitudinal commands and for lateral 
command. These gains are set in accordance with curves 
illustrated in Figure 50. With the use of such gains, the 
controller can a/locate commands for prompting the operator to 
extensively coping with risks in an angular location where the 
25 combined risk originates. 

While the present invention has been particularly 
described, in conjunction with various implementations of the 
present invention, it is evident that many alternatives, 
modifications and variations will be apparent to those skilled in 
30 the art in light of the foregoing description. It is therefore 
contemplated that the appended claims will embrace any such 
alternatives, modifications and variations as faffing within the 
true scope and spirit of the present invention. 
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This application claims the priorities of Japanese Patent 
Applications No. P2001-252422, filed August 23, 2001, No. 
P2001-389314, filed December 21, 2001, No. P2002-138266, 
filed May 14, 2002, and No. P2002-1 77029, Filed June 18, 2002, 
s the disclosure of each of which is hereby incorporated by 
reference in its entirety. 



